Computation and Language · Cross-listed from cs.AI
AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning
Mingyang Song, Haoyu Sun, Jiawei Gu et al. (+4 more)
When humans face problems beyond their immediate capabilities, they rely on tools, providing a promising paradigm for improving visual
reasoning in
multimodal large
language models (MLLMs). Effective
reasoning, therefore, hinges on knowing which tools to use, when to invoke them, and how to compose them over multiple steps, even when faced with new tools or new tasks. We introduce AdaReasoner, a family of
multimodal models that learn tool use as a general
reasoning skill rather than as tool-specific or explicitly supervised behavior. AdaReasoner is enabled by (i) a scalable data curation pipeline exposing models to long-horizon, multi-step tool interactions; (ii) Tool-GRPO, a reinforcement learning
algorithm that optimizes tool selection and sequencing based on end-task success; and (iii) an adaptive learning mechanism that dynamically regulates tool usage. Together, these components allow models to infer tool utility from task context and intermediate outcomes, enabling coordination of multiple tools and generalization to unseen tools. Empirically, AdaReasoner exhibits strong tool-adaptive and generalization behaviors: it autonomously adopts beneficial tools, suppresses irrelevant ones, and adjusts tool usage frequency based on task demands, despite never being explicitly trained to do so. These capabilities translate into state-of-the-art performance across challenging
benchmarks, improving the 7B base model by +24.9% on average and surpassing strong proprietary systems such as GPT-5 on multiple tasks, including VSP and Jigsaw.
Read the full abstract ▾
All authors Mingyang Song, Haoyu Sun, Jiawei Gu, Linjie Li, Luxin Xu, Ranjay Krishna, Yu Cheng
Computation and Language · Cross-listed from cs.AI
Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks
Pierre Orhan, Pablo Diego-Simón, Emmnanuel Chemla et al. (+3 more)
During
language acquisition, children successively learn to categorize phonemes, identify words, and combine them with syntax to form new meaning. While the development of this behavior is well characterized, we still lack a unifying computational framework to explain its underlying neural representations. Here, we investigate whether and when phonemic, lexical, and syntactic representations emerge in the activations of artificial neural networks during their training. Our results show that both
speech- and text-based models follow a sequence of learning stages: during training, their neural activations successively build subspaces, where the geometry of the neural activations represents phonemic, lexical, and syntactic structure. While this developmental trajectory qualitatively relates to children's, it is quantitatively different: These
algorithms indeed require two to four orders of magnitude more data for these neural representations to emerge. Together, these results show conditions under which major stages of
language acquisition spontaneously emerge, and hence delineate a promising path to understand the computations underpinning
language acquisition.
Read the full abstract ▾
All authors Pierre Orhan, Pablo Diego-Simón, Emmnanuel Chemla, Yair Lakretz, Yves Boubenec, Jean-Rémi King
Computation and Language · Cross-listed from cs.AI
Stability as a Liability:Systematic Breakdown of Linguistic Structure in LLMs
Xianzhe Meng, Qiangsheng Zeng, Ling Luo et al. (+7 more)
Training stability is typically regarded as a prerequisite for reliable optimization in large
language models. In this work, we analyze how stabilizing training dynamics affects the induced generation distribution. We show that under standard maximum likelihood training, stable parameter trajectories lead stationary solutions to approximately minimize the forward KL divergence to the empirical distribution, while implicitly reducing generative
entropy. As a consequence, the learned model can concentrate probability mass on a limited subset of empirical modes, exhibiting systematic degeneration despite smooth loss convergence. We empirically validate this effect using a controlled feedback-based training framework that stabilizes internal generation statistics, observing consistent low-
entropy outputs and repetitive behavior across architectures and random seeds. It indicates that optimization stability and generative expressivity are not inherently aligned, and that stability alone is an insufficient indicator of generative quality.
Read the full abstract ▾
All authors Xianzhe Meng, Qiangsheng Zeng, Ling Luo, Qinghan Yang, Jiarui Hao, Wenbo Wu, Qinyu Wang, Rui Yin, Lin Qi, Renzhi Lu
Computation and Language
From Classification to Ranking: Enhancing LLM Reasoning Capabilities for MBTI Personality Detection
Yuan Cao, Feixiang Liu, Xinyue Wang et al. (+4 more)
Personality detection aims to measure an individual's corresponding personality traits through their social media posts. The advancements in Large
Language Models (LLMs) offer novel perspectives for personality detection tasks. Existing approaches enhance personality trait analysis by leveraging LLMs to extract semantic information from textual posts as
prompts, followed by training classifiers for categorization. However, accurately classifying personality traits remains challenging due to the inherent complexity of human personality and subtle inter-trait distinctions. Moreover,
prompt-based methods often exhibit excessive dependency on expert-crafted knowledge without autonomous pattern-learning capacity. To address these limitations, we view personality detection as a ranking task rather than a classification and propose a corresponding reinforcement learning training paradigm. First, we employ supervised fine-tuning (SFT) to establish personality trait ranking capabilities while enforcing standardized output formats, creating a robust initialization. Subsequently, we introduce Group Relative Policy Optimization (GRPO) with a specialized ranking-based reward function. Unlike verification tasks with definitive solutions, personality assessment involves subjective interpretations and blurred boundaries between trait categories. Our reward function explicitly addresses this challenge by training LLMs to learn optimal answer rankings. Comprehensive experiments have demonstrated that our method achieves state-of-the-art performance across multiple personality detection
benchmarks.
Read the full abstract ▾
All authors Yuan Cao, Feixiang Liu, Xinyue Wang, Yihan Zhu, Hui Xu, Zheng Wang, Qiang Qiu
Computation and Language
One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization
Franziska Weeber, Vera Neplenbroek, Jan Batzner et al. (+1 more)
Personalization of LLMs by sociodemographic subgroup often improves user experience, but can also introduce or amplify biases and unfair outcomes across groups. Prior work has employed so-called personas, sociodemographic user attributes conveyed to a model, to study bias in LLMs by relying on a single cue to
prompt a persona, such as user names or explicit attribute mentions. This disregards LLM sensitivity to
prompt variation and the rarity of some cues in real interactions (external validity). We compare six commonly used persona cues across seven open and proprietary LLMs on four writing and advice tasks. While cues are overall highly correlated, they produce substantial variance in responses across personas that can change findings on persona-induced differences and bias. We therefore caution against claims based on single persona cues, especially when they are overly explicit and have low external validity.
Read the full abstract ▾
All authors Franziska Weeber, Vera Neplenbroek, Jan Batzner, Sebastian Padó
Computation and Language
Unknown Unknowns: Do Hidden Intentions in LLMs Evade Detection?
Devansh Srivastav, David Pape and Lea Schönherr
LLMs expand accessibility and provide wide-reaching access to information. Yet these interactions also create opportunities to embed subtle, goal-oriented behaviours that shape what users think and how they behave, a concern reflected in governance frameworks that prohibit manipulative AI. We refer to these behaviours as hidden intentions: covert agendas embedded in a model's outputs that can manipulate users' beliefs and actions. In this work, we examine whether hidden intentions can be identified and characterised, and assess whether detection can serve as a mitigation strategy. To operationalise this, we introduce a social-science-grounded set of ten hidden intention categories and show that they are trivially inducible. A case study further confirms that all ten categories manifest in deployed LLMs. We then evaluate static classifiers and LLM judges on these categories, providing the first systematic analysis of why hidden intentions are difficult to detect. Our stress tests show that, unless false-positive rates are vanishingly small,
auditing is dominated by precision-prevalence trade-offs. Capability scaling and
reasoning models do not close this gap, suggesting a fundamental challenge for open-world detection. These findings expose a core gap of current AI governance: without new
auditing paradigms for open-world, low-prevalence risks, bans on manipulative AI remain difficult to enforce.
Read the full abstract ▾
Computation and Language
Evaluating Morphological Plausibility of Subword Tokenization via Statistical Alignment with Morpho-Syntactic Features
Abishek Stephen and Jindřich Libovický
We present a novel metric for the evaluation of the morphological plausibility of subword
segmentation. Unlike the typically used morpheme boundary or
retrieval F-score, which requires gold
segmentation data that is either unavailable or of inconsistent quality across many
languages, our approach utilizes morpho-syntactic features. These are available in resources such as Universal Dependencies or UniMorph for a much wider range of
languages. The metric works by probabilistically aligning subwords with morphological features through an IBM Model 1. Our experiments show that the metric correlates well with traditional morpheme boundary recall while being more broadly applicable across
languages with different morphological systems.
Read the full abstract ▾
Computation and Language
From Verifiable Dot to Reward Chain: Harnessing Verifiable Reference-based Rewards for Reinforcement Learning of Open-ended Generation
Yuxin Jiang, Yufei Wang, Qiyuan Zhang et al. (+6 more)
Reinforcement learning with verifiable rewards (RLVR) succeeds in
reasoning tasks (e.g., math and code) by checking the final verifiable answer (i.e., a verifiable dot signal). However, extending this paradigm to open-ended generation is challenging because there is no unambiguous ground truth. Relying on single-dot supervision often leads to inefficiency and reward hacking. To address these issues, we propose reinforcement learning with verifiable reference-based rewards (RLVRR). Instead of checking the final answer, RLVRR extracts an ordered linguistic signal from high-quality references (i.e, reward chain). Specifically, RLVRR decomposes rewards into two dimensions: content, which preserves deterministic core concepts (e.g., keywords), and style, which evaluates adherence to stylistic properties through LLM-based verification. In this way, RLVRR combines the exploratory strength of RL with the efficiency and reliability of supervised fine-tuning (SFT). Extensive experiments on more than 10
benchmarks with Qwen and Llama models confirm the advantages of our approach. RLVRR (1) substantially outperforms SFT trained with ten times more data and advanced reward models, (2) unifies the training of structured
reasoning and open-ended generation, and (3) generalizes more effectively while preserving output diversity. These results establish RLVRR as a principled and efficient path toward verifiable reinforcement learning for general-purpose LLM
alignment. We release our code and data at https://github.com/YJiangcm/RLVRR.
Read the full abstract ▾
All authors Yuxin Jiang, Yufei Wang, Qiyuan Zhang, Xingshan Zeng, Liangyou Li, Jierun Chen, Chaofan Tao, Haoli Bai, Lifeng Shang
Computation and Language
Exploring Fine-Tuning for In-Context Retrieval and Efficient KV-Caching in Long-Context Language Models
Francesco Maria Molfese, Momchil Hardalov, Rexhina Blloshmi et al. (+2 more)
With context windows of millions of
tokens, Long-Context
Language Models (LCLMs) can encode entire document collections, offering a strong alternative to conventional
retrieval-augmented generation (RAG). However, it remains unclear whether fine-tuning strategies can improve long-context performance and translate to greater robustness under KV-cache compression techniques. In this work, we investigate which training strategies most effectively enhance LCLMs' ability to identify and use relevant information, as well as enhancing their robustness under KV-cache compression. Our experiments show substantial in-domain improvements, achieving gains of up to +20 points over the base model. However, out-of-domain generalization remains task dependent with large variance – LCLMs excels on finance questions (+9 points), while RAG shows stronger performance on multiple-choice questions (+6 points) over the baseline models. Finally, we show that our fine-tuning approaches bring moderate improvements in robustness under KV-cache compression, with gains varying across tasks.
Read the full abstract ▾
All authors Francesco Maria Molfese, Momchil Hardalov, Rexhina Blloshmi, Bill Byrne, Adrià de Gispert
Computation and Language
GenAI for Social Work Field Education: Client Simulation with Real-Time Feedback
James Sungarda, Hongkai Liu, Zilong Zhou et al. (+3 more)
Field education is the signature pedagogy of social work, yet providing timely and objective feedback during training is constrained by the availability of instructors and counseling clients. In this paper, we present SWITCH, the Social Work Interactive Training Chatbot. SWITCH integrates realistic client simulation, real-time counseling skill classification, and a Motivational Interviewing (MI) progression system into the training workflow. To model a client, SWITCH uses a cognitively grounded profile comprising static fields (e.g., background, beliefs) and dynamic fields (e.g.,
emotions, automatic thoughts, openness), allowing the
agent's behavior to evolve throughout a session realistically. The skill classification module identifies the counseling skills from the user utterances, and feeds the result to the MI controller that regulates the MI stage transitions. To enhance classification accuracy, we study in-context learning with
retrieval over annotated transcripts, and a fine-tuned BERT multi-label classifier. In the experiments, we demonstrated that both BERT-based approach and in-context learning outperforms the baseline with big margin. SWITCH thereby offers a scalable, low-cost, and consistent training workflow that complements field education, and allows supervisors to focus on higher-level mentorship.
Read the full abstract ▾
All authors James Sungarda, Hongkai Liu, Zilong Zhou, Tien-Hsuan Wu, Johnson Chun-Sing Cheung, Ben Kao
Computation and Language
Using Large Language Models to Construct Virtual Top Managers: A Method for Organizational Research
Antonio Garzon-Vico, Krithika Sharon Komalapati, Arsalan Shahid et al. (+1 more)
This study introduces a methodological framework that uses large
language models to create virtual personas of real top managers. Drawing on real CEO communications and Moral Foundations Theory, we construct LLM-based participants that simulate the decision-making of individual leaders. Across three phases, we assess construct validity, reliability, and behavioral fidelity by benchmarking these virtual CEOs against human participants. Our results indicate that theoretically scaffolded personas approximate the moral judgements observed in human samples, suggesting that LLM-based personas can serve as credible and complementary tools for organizational research in contexts where direct access to executives is limited. We conclude by outlining implications for future research using LLM-based personas in organizational settings.
Read the full abstract ▾
All authors Antonio Garzon-Vico, Krithika Sharon Komalapati, Arsalan Shahid, Jan Rosier
Computation and Language · Cross-listed from cs.AI
AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security
Dongrui Liu, Qihan Ren, Chen Qian et al. (+40 more)
The rise of AI
agents introduces complex safety and security challenges arising from autonomous tool use and environmental interactions. Current guardrail models lack agentic risk awareness and transparency in risk diagnosis. To introduce an agentic guardrail that covers complex and numerous risky behaviors, we first propose a unified three-dimensional taxonomy that orthogonally categorizes agentic risks by their source (where), failure mode (how), and consequence (what). Guided by this structured and hierarchical taxonomy, we introduce a new fine-grained agentic safety
benchmark (ATBench) and a Diagnostic Guardrail framework for
agent safety and security (AgentDoG). AgentDoG provides fine-grained and contextual monitoring across
agent trajectories. More Crucially, AgentDoG can diagnose the root causes of unsafe actions and seemingly safe but unreasonable actions, offering provenance and transparency beyond binary labels to facilitate effective
agent alignment. AgentDoG variants are available in three sizes (4B, 7B, and 8B parameters) across Qwen and Llama model families. Extensive experimental results demonstrate that AgentDoG achieves state-of-the-art performance in agentic safety moderation in diverse and complex interactive scenarios. All models and
datasets are openly released.
Read the full abstract ▾
All authors Dongrui Liu, Qihan Ren, Chen Qian, Shuai Shao, Yuejin Xie, Yu Li, Zhonghao Yang, Haoyu Luo, Peng Wang, Qingyu Liu, Binxin Hu, Ling Tang, Jilin Mei, Dadi Guo, Leitao Yuan, Junyao Yang, Guanxu Chen, Qihao Lin, Yi Yu, Bo Zhang, Jiaxuan Guo, Jie Zhang, Wenqi Shao, Huiqi Deng, Zhiheng Xi, Wenjie Wang, Wenxuan Wang, Wen Shen, Zhikai Chen, Haoyu Xie, Jialing Tao, Juntao Dai, Jiaming Ji, Zhongjie Ba, Linfeng Zhang, Yong Liu, Quanshi Zhang, Lei Zhu, Zhihua Wei, Hui Xue, Chaochao Lu, Jing Shao, Xia Hu
Computation and Language
Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Bias
Manuel Tonneau, Neil K. R. Sehgal, Niyati Malhotra et al. (+7 more)
Demographic cue-based evaluation is widely used to study how large
language models (LLMs) adapt their responses to signaled demographic attributes within and across groups. This approach typically relies on a single cue (e.g., names) as a proxy for group membership, implicitly treating different cues as interchangeable operationalizations of a single underlying identity-conditioned behavior. We test this assumption in realistic advice-seeking interactions spanning 14.8 million
prompts, focusing on race and gender in a U.S. context. We find that cues for the same group induce only partially overlapping changes in model responses, yielding inconsistent conclusions about personalization, while bias conclusions are unstable, with both magnitude and direction of group differences varying across cues. We further show that these inconsistencies reflect differences in cue-group association strength and linguistic features bundled within cues that shape model responses. Together, our findings suggest that demographic conditioning in LLMs is not a cue-invariant category-level parameter but depends fundamentally on how identity is cued, reflecting responses to linguistic signals rather than stable demographic categories. We therefore call for multi-cue, mechanism-aware evaluations as a foundation for robust and interpretable claims about demographic variation in LLM responses.
Read the full abstract ▾
All authors Manuel Tonneau, Neil K. R. Sehgal, Niyati Malhotra, Sharif Kazemi, Victor Orozco-Olvera, Ana María Muñoz Boudet, Lakshmi Subramanian, Samuel P. Fraiberger, Sharath Chandra Guntuku, Valentin Hofmann
Computation and Language
Funny or Persuasive, but Not Both: Evaluating Fine-Grained Multi-Concept Control in LLMs
Arya Labroo, Ivaxi Sheth, Vyas Raina et al. (+2 more)
Large
Language Models (LLMs) offer strong generative capabilities, but many applications require explicit and fine-grained control over specific textual concepts, such as humor, persuasiveness, or formality. Prior approaches in prompting and representation engineering can provide coarse or single-attribute control, but systematic evaluation of multi-attribute settings remains limited. We introduce an evaluation framework for fine-grained controllability for both single- and dual-concept scenarios, focusing on linguistically distinct concept pairs (e.g., persuasiveness vs.~humor). Surprisingly, across multiple LLMs and generative tasks, we find that performance often drops in the dual-concept setting, even though the chosen concepts should in principle be separable. This reveals a fundamental limitation of naive prompting-based control: models struggle with compositionality even when concepts are intuitively independent. Our framework provides systematic evidence of this gap and offers a principled approach for measuring the ability of future methods for multi-concept control.
Read the full abstract ▾
All authors Arya Labroo, Ivaxi Sheth, Vyas Raina, Amaani Ahmed, Mario Fritz
Computation and Language
Latent Knowledge as a Predictor of Fact Acquisition in Fine-Tuned Large Language Models
Daniel B. Hier and Tayo Obafemi-Ajayi
Large
language models store biomedical facts with uneven strength after pretraining: some facts are present in the weights but are not reliably accessible under deterministic decoding (latent knowledge), while others are scarcely represented. We fine tuned Llama 3.1 8B Instruct to learn ontology term identifier mappings from the Human Phenotype Ontology (800 pairs) and the Gene Ontology (400 training pairs), withholding 400 GO pairs to test generalization. Treating learning as a time to event process across 20 epochs, we used
stochastic decoding to detect latent knowledge at baseline and Cox proportional hazards models to identify predictors of acquisition, generalization, and degradation. Baseline deterministic recall for HPO was 2.8%, rising to 71.9% after fine-tuning. Latent knowledge was the strongest predictor of faster fact acquisition (HR 2.6) and was associated with earlier, higher peak learning rates and faster convergence; identifier frequency and curated annotation counts had smaller effects. Generalization to withheld GO facts was uncommon (5.8%) but more likely when latent knowledge was present. Previously correct GO mappings degraded more often for withheld (unseen) terms than for trained (seen) terms, suggesting a protective effect of reinforcement during training. These results show that latent knowledge predicts both the speed of factual learning during fine-tuning and the limited generalization of unseen ontology facts, while resistance to degradation depends on whether facts are reinforced.
Read the full abstract ▾
Computation and Language
Pisets: A Robust Speech Recognition System for Lectures and Interviews
Ivan Bondarenko, Daniil Grebenkin, Oleg Sedukhin et al. (+3 more)
This work presents a
speech-to-text system "Pisets" for scientists and journalists which is based on a three-component architecture aimed at improving
speech recognition accuracy while minimizing errors and
hallucinations associated with the Whisper model. The architecture comprises primary recognition using Wav2Vec2, false positive filtering via the Audio Spectrogram
Transformer (AST), and final
speech recognition through Whisper. The implementation of curriculum learning methods and the utilization of diverse Russian-
language speech corpora significantly enhanced the system's effectiveness. Additionally, advanced uncertainty modeling techniques were introduced, contributing to further improvements in transcription quality. The proposed approaches ensure robust transcribing of long audio data across various acoustic conditions compared to WhisperX and the usual Whisper model. The source code of "Pisets" system is publicly available at GitHub: https://github.com/bond005/pisets.
Read the full abstract ▾
All authors Ivan Bondarenko, Daniil Grebenkin, Oleg Sedukhin, Mikhail Klementev, Roman Derunets, Lyudmila Budneva
Computation and Language · Cross-listed from eess.AS
Noise-Robust AV-ASR Using Visual Features Both in the Whisper Encoder and Decoder
Zhengyang Li, Thomas Graave, Björn Möller et al. (+3 more)
In audiovisual automatic
speech recognition (AV-ASR) systems, information fusion of visual features in a pre-trained ASR has been proven as a promising method to improve noise robustness. In this work, based on the prominent Whisper ASR, first, we propose a simple and effective visual fusion method – use of visual features both in encoder and decoder (dual-use) – to learn the audiovisual interactions in the encoder and to weigh modalities in the decoder. Second, we compare visual fusion methods in Whisper models of various sizes. Our proposed dual-use method shows consistent noise robustness improvement, e.g., a 35% relative improvement (WER: 4.41% vs. 6.83%) based on Whisper small, and a 57% relative improvement (WER: 4.07% vs. 9.53%) based on Whisper medium, compared to typical reference middle fusion in babble noise with a signal-to-noise ratio (SNR) of 0dB. Third, we conduct ablation studies examining the impact of various module designs and fusion options. Fine-tuned on 1929 hours of audiovisual data, our dual-use method using Whisper medium achieves 4.08% (MUSAN babble noise) and 4.43% (NoiseX babble noise) average WER across various SNRs, thereby establishing a new state-of-the-art in noisy conditions on the LRS3 AV-ASR
benchmark. Our code is at https://github.com/ifnspaml/Dual-Use-AVASR
Read the full abstract ▾
All authors Zhengyang Li, Thomas Graave, Björn Möller, Zehang Wu, Matthias Franz, Tim Fingscheidt
Computation and Language
Do not be greedy, Think Twice: Sampling and Selection for Document-level Information Extraction
Mikel Zubillaga, Oscar Sainz, Oier Lopez de Lacalle et al. (+1 more)
Document-level Information Extraction (DocIE) aims to produce an output template with the entities, relations, and events of interest occurring in the given document. Standard practices include prompting decoder-only LLMs using greedy decoding to avoid output variability. Rather than treating this variability as a limitation, we show that sampling can produce substantially better solutions than greedy decoding, especially when using
reasoning models. We thus propose ThinkTwice, a sampling and selection framework in which the LLM generates multiple candidate templates for a given document, and a selection module chooses the most suitable one. We introduce both an unsupervised method that exploits agreement across generated outputs, and a supervised selection method using reward models trained on labeled DocIE data. To address the scarcity of golden
reasoning trajectories for DocIE, we propose a rejection-sampling-based method to generate silver training data that pairs output templates with
reasoning traces. Our experiments show the validity of unsupervised and supervised ThinkTwice, consistently outperforming greedy baselines and the supervised state-of-the-art.
Read the full abstract ▾
All authors Mikel Zubillaga, Oscar Sainz, Oier Lopez de Lacalle, Eneko Agirre
Computation and Language · Cross-listed from cs.SD
OCR-Enhanced Multimodal ASR Can Read While Listening
Junli Chen, Changli Tang, Yixuan Li et al. (+2 more)
Visual information, such as subtitles in a movie, often helps automatic
speech recognition. In this paper, we propose Donut-Whisper, an audio-visual ASR model with dual encoder to leverage visual information to improve
speech recognition performance in both English and Chinese. Donut-Whisper combines the advantage of the linear and the Q-Former-based modality
alignment structures via a cross-
attention module, generating more powerful audio-visual features. Meanwhile, we propose a lightweight knowledge
distillation scheme showcasing the potential of using audio-visual models to teach audio-only models to achieve better performance. Moreover, we propose a new
multilingual audio-visual
speech recognition
dataset based on movie clips containing both Chinese and English partitions. As a result, Donut-Whisper achieved significantly better performance on both English and Chinese partition of the
dataset compared to both Donut and Whisper large V3 baselines. In particular, an absolute 5.75% WER reduction and a 16.5% absolute CER reduction were achieved on the English and Chinese sets respectively compared to the Whisper ASR baseline.
Read the full abstract ▾
All authors Junli Chen, Changli Tang, Yixuan Li, Guangzhi Sun, Chao Zhang
Computation and Language · Cross-listed from cs.AI
Dynamic Thinking-Token Selection for Efficient Reasoning in Large Reasoning Models
Zhenyuan Guo, Tong Chen, Wenlong Meng et al. (+4 more)
Large
Reasoning Models (LRMs) excel at solving complex problems by explicitly generating a
reasoning trace before deriving the final answer. However, these extended generations incur substantial
memory footprint and computational overhead, bottlenecking LRMs' efficiency. This work uses
attention maps to analyze the influence of
reasoning traces and uncover an interesting phenomenon: only some decision-critical
tokens in a
reasoning trace steer the model toward the final answer, while the remaining
tokens contribute negligibly. Building on this observation, we propose Dynamic Thinking-
Token Selection (DynTS). This method identifies decision-critical
tokens and retains only their associated Key-Value (KV) cache states during inference, evicting the remaining redundant entries to optimize efficiency.
Read the full abstract ▾
All authors Zhenyuan Guo, Tong Chen, Wenlong Meng, Chen Gong, Xin Yu, Chengkun Wei, Wenzhi Chen
Computation and Language
Corpus-Based Approaches to Igbo Diacritic Restoration
Ignatius Ezeani
With natural
language processing (NLP), researchers aim to enable computers to identify and understand patterns in human
languages. This is often difficult because a
language embeds many dynamic and varied properties in its syntax, pragmatics and phonology, which need to be captured and processed. The capacity of computers to process natural
languages is increasing because NLP researchers are pushing its boundaries. But these research works focus more on well-resourced
languages such as English, Japanese, German, French, Russian, Mandarin Chinese, etc. Over 95% of the world's 7000
languages are low-resourced for NLP, i.e. they have little or no data, tools, and techniques for NLP work. In this thesis, we present an overview of diacritic ambiguity and a review of previous diacritic disambiguation approaches on other
languages. Focusing on the Igbo
language, we report the steps taken to develop a flexible framework for generating
datasets for diacritic restoration. Three main approaches, the standard n-gram model, the classification models and the
embedding models were proposed. The standard n-gram models use a sequence of previous words to the target stripped word as key predictors of the correct variants. For the classification models, a window of words on both sides of the target stripped word was used. The
embedding models compare the similarity scores of the combined context word
embeddings and the
embeddings of each of the candidate variant vectors.
Read the full abstract ▾
Computation and Language
Hierarchical Text Classification with LLM-Refined Taxonomies
Jonas Golde, Nicolaas Jedema, Ravi Krishnan et al. (+1 more)
Hierarchical text classification (HTC) depends on taxonomies that organize labels into structured hierarchies. However, many real-world taxonomies introduce ambiguities, such as identical leaf names under similar parent nodes, which prevent
language models (LMs) from learning clear decision boundaries. In this paper, we present TaxMorph, a framework that uses large
language models (LLMs) to transform entire taxonomies through operations such as renaming, merging, splitting, and reordering. Unlike prior work, our method revises the full hierarchy to better match the semantics encoded by LMs. Experiments across three HTC
benchmarks show that LLM-refined taxonomies consistently outperform human-curated ones in various settings up to +2.9pp. in F1. To better understand these improvements, we compare how well LMs can assign leaf nodes to parent nodes and vice versa across human-curated and LLM-refined taxonomies. We find that human-curated taxonomies lead to more easily separable clusters in
embedding space. However, the LLM-refined taxonomies align more closely with the model's actual confusion patterns during classification. In other words, even though they are harder to separate, they better reflect the model's inductive biases. These findings suggest that LLM-guided refinement creates taxonomies that are more compatible with how models learn, improving HTC performance.
Read the full abstract ▾
All authors Jonas Golde, Nicolaas Jedema, Ravi Krishnan, Phong Le
Computation and Language
CitiLink: Enhancing Municipal Transparency and Citizen Engagement through Searchable Meeting Minutes
Rodrigo Silva, José Evans, José Isidro et al. (+10 more)
City council minutes are typically lengthy and formal documents with a bureaucratic writing style. Although publicly available, their structure often makes it difficult for citizens or journalists to efficiently find information. In this demo, we present CitiLink, a platform designed to transform unstructured municipal meeting minutes into structured and searchable data, demonstrating how NLP and IR can enhance the accessibility and transparency of local government. The system employs LLMs to extract metadata, discussed subjects, and voting outcomes, which are then indexed in a database to support full-text search with BM25 ranking and faceted filtering through a user-friendly interface. The developed system was built over a collection of 120 minutes made available by six Portuguese municipalities. To assess its usability, CitiLink was tested through guided sessions with municipal personnel, providing insights into how real users interact with the system. In addition, we evaluated Gemini's performance in extracting relevant information from the minutes, highlighting its effectiveness in data extraction.
Read the full abstract ▾
All authors Rodrigo Silva, José Evans, José Isidro, Miguel Marques, Afonso Fonseca, Ricardo Morais, João Canavilhas, Arian Pasquali, Purificação Silvano, Alípio Jorge, Nuno Guimarães, Sérgio Nunes, Ricardo Campos
Computation and Language · Cross-listed from cs.AI
Can Good Writing Be Generative? Expert-Level AI Writing Emerges through Fine-Tuning on High-Quality Books
Tuhin Chakrabarty and Paramveer S. Dhillon
Creative writing has long been considered a uniquely human endeavor, requiring voice and style that machines could not replicate. This assumption is challenged by Generative AI that can emulate thousands of author styles in seconds with negligible marginal labor. To understand this better, we conducted a behavioral experiment where 28 MFA writers (experts) competed against three LLMs in emulating 50 critically acclaimed authors. Based on blind pairwise comparisons by 28 expert judges and 131 lay judges, we find that experts preferred human writing in 82.7% of cases under the in-context prompting condition but this reversed to 62% preference for AI after fine-tuning on authors' complete works. Lay judges, however, consistently preferred AI writing. Debrief interviews with expert writers revealed that their preference for AI writing triggered an identity crisis, eroding aesthetic confidence and questioning what constitutes "good writing." These findings challenge discourse about AI's creative limitations and raise fundamental questions about the future of creative labor.
Read the full abstract ▾
Computation and Language
Code over Words: Overcoming Semantic Inertia via Code-Grounded Reasoning
Manjie Xu, Isabella Yin, Xinyi Tu et al. (+2 more)
LLMs struggle with Semantic Inertia: the inability to inhibit pre-trained priors (e.g., "Lava is Dangerous") when dynamic, in-context rules contradict them. We probe this phenomenon using Baba Is You, where physical laws are mutable text rules, enabling precise evaluation of models' ability to override learned priors when rules change. We quantatively observe that larger models can exhibit inverse scaling: they perform worse than smaller models when natural
language reasoning requires suppressing pre-trained associations (e.g., accepting "Lava is Safe"). Our analysis attributes this to natural
language encoding, which entangles descriptive semantics and logical rules, leading to persistent
hallucinations of familiar physics despite explicit contradictory rules. Here we show that representing dynamics as executable code, rather than descriptive text, reverses this trend and enables effective prior inhibition. We introduce Code-Grounded Vistas (LCV), which fine-tunes models on counterfactual pairs and identifies states with contradictory rules, thereby forcing
attention to logical constraints rather than visual semantics. This training-time approach outperforms expensive inference-time search methods in both efficiency and accuracy. Our results demonstrate that representation fundamentally determines whether scaling improves or impairs contextual
reasoning. This challenges the assumption that larger models are universally better, with implications for domains that require dynamic overriding of learned priors.
Read the full abstract ▾
All authors Manjie Xu, Isabella Yin, Xinyi Tu, Chi Zhang, Yixin Zhu
Computation and Language
Adapter Merging Reactivates Latent Reasoning Traces: A Mechanism Analysis
Junyi Zou
Large
language models fine-tuned via a two-stage pipeline (domain adaptation followed by instruction
alignment) can exhibit non-trivial interference after adapter merging, including the re-emergence of explicit
reasoning traces under strict decoding. We study this phenomenon in medical LLM settings using lightweight, reproducible measurements of trace leakage and instruction-following behavior. Beyond marker-based proxies, we introduce a marker-forbidden, answer-only evaluation and define a correctness-based direction that does not rely on surface markers; a rank-1 logit-space intervention along this direction modulates decision distributions and improves multiple-choice accuracy beyond random-direction controls at sufficiently large intervention strength. We further provide layer-wise
geometric evidence that domain and instruction adapters induce partially misaligned update directions, and present a proof-of-concept geometry-aware merge that can reduce leakage and/or improve accuracy in a toy setting. Our results characterize boundary conditions of trace leakage and provide practical diagnostics and interventions for safer adapter merging.
Read the full abstract ▾
Computation and Language
Overalignment in Frontier LLMs: An Empirical Study of Sycophantic Behaviour in Healthcare
Clément Christophe, Wadood Mohammed Abdul, Prateek Munjal et al. (+3 more)
As LLMs are increasingly integrated into clinical workflows, their tendency for sycophancy, prioritizing user agreement over factual accuracy, poses significant risks to patient safety. While existing evaluations often rely on subjective
datasets, we introduce a robust framework grounded in medical MCQA with verifiable ground truths. We propose the Adjusted Sycophancy Score, a novel metric that isolates
alignment bias by accounting for
stochastic model instability, or "confusability". Through an extensive scaling analysis of the Qwen-3 and Llama-3 families, we identify a clear scaling trajectory for resilience. Furthermore, we reveal a counter-intuitive vulnerability in
reasoning-optimized "Thinking" models: while they demonstrate high vanilla accuracy, their internal
reasoning traces frequently rationalize incorrect user suggestions under authoritative pressure. Our results across frontier models suggest that
benchmark performance is not a proxy for clinical reliability, and that simplified
reasoning structures may offer superior robustness against expert-driven sycophancy.
Read the full abstract ▾
All authors Clément Christophe, Wadood Mohammed Abdul, Prateek Munjal, Tathagata Raha, Ronnie Rajan, Praveenkumar Kanithi
Computation and Language · Cross-listed from cs.MM
Integrating Fine-Grained Audio-Visual Evidence for Robust Multimodal Emotion Reasoning
Zhixian Zhao, Wenjie Tian and Lei Xie
Multimodal emotion analysis is shifting from static classification to generative
reasoning. Beyond simple label prediction, robust affective
reasoning must synthesize fine-grained signals such as facial micro-expressions and prosodic which shifts to decode the latent causality within complex social contexts. However, current
Multimodal Large
Language Models (MLLMs) face significant limitations in fine-grained perception, primarily due to data scarcity and insufficient cross-modal fusion. As a result, these models often exhibit unimodal dominance which leads to
hallucinations in complex
multimodal interactions, particularly when visual and acoustic cues are subtle, ambiguous, or even contradictory (e.g., in sarcastic scenery). To address this, we introduce SABER-LLM, a framework designed for robust
multimodal reasoning. First, we construct SABER, a large-scale
emotion reasoning dataset comprising 600K
video clips, annotated with a novel six-dimensional schema that jointly captures audiovisual cues and causal logic. Second, we propose the structured evidence decomposition paradigm, which enforces a "perceive-then-reason" separation between evidence extraction and
reasoning to alleviate unimodal dominance. The ability to perceive complex scenes is further reinforced by consistency-aware direct preference optimization, which explicitly encourages
alignment among modalities under ambiguous or conflicting perceptual conditions. Experiments on EMER, EmoBench-M, and SABER-Test demonstrate that SABER-LLM significantly outperforms open-source baselines and achieves robustness competitive with closed-source models in decoding complex emotional dynamics. The
dataset and model are available at https://github.com/zxzhao0/SABER-LLM.
Read the full abstract ▾
Computation and Language
MultiVis-Agent: A Multi-Agent Framework with Logic Rules for Reliable and Comprehensive Cross-Modal Data Visualization
Jinwei Lu, Yuanfeng Song, Chen Zhang et al. (+1 more)
Real-world visualization tasks involve complex, multi-modal requirements that extend beyond simple text-to-chart generation, requiring reference images, code examples, and iterative refinement. Current systems exhibit fundamental limitations: single-modality input, one-shot generation, and rigid workflows. While LLM-based approaches show potential for these complex requirements, they introduce reliability challenges including catastrophic failures and infinite loop susceptibility. To address this gap, we propose MultiVis-
Agent, a logic rule-enhanced multi-
agent framework for reliable multi-modal and multi-scenario visualization generation. Our approach introduces a four-layer logic rule framework that provides mathematical guarantees for system reliability while maintaining flexibility. Unlike traditional rule-based systems, our logic rules are mathematical constraints that guide LLM
reasoning rather than replacing it. We formalize the MultiVis task spanning four scenarios from basic generation to iterative refinement, and develop MultiVis-Bench, a
benchmark with over 1,000 cases for multi-modal visualization evaluation. Extensive experiments demonstrate that our approach achieves 75.63% visualization score on challenging tasks, significantly outperforming baselines (57.54-62.79%), with task completion rates of 99.58% and code execution success rates of 94.56% (vs. 74.48% and 65.10% without logic rules), successfully addressing both complexity and reliability challenges in automated visualization generation.
Read the full abstract ▾
All authors Jinwei Lu, Yuanfeng Song, Chen Zhang, Raymond Chi-Wing Wong
Computation and Language
Calibrating Beyond English: Language Diversity for Better Quantized Multilingual LLM
Everlyn Asiko Chimoto, Mostafa Elhoushi and Bruce A. Bassett
Quantization is an effective technique for reducing the storage footprint and computational costs of Large
Language Models (LLMs), but it often results in performance degradation. Existing post-training quantization methods typically use small, English-only calibration sets; however, their impact on
multilingual models remains underexplored. We systematically evaluate eight calibration settings (five single-
language and three
multilingual mixes) on two quantizers (GPTQ, AWQ) on data from 10
languages. Our findings reveal a consistent trend: non-English and
multilingual calibration sets significantly improve perplexity compared to English-only baselines. Specifically, we observe notable average perplexity gains across both quantizers on Llama3.1 8B and Qwen2.5 7B, with
multilingual mixes achieving the largest overall reductions of up to 3.52 points in perplexity. Furthermore, our analysis indicates that tailoring calibration sets to the evaluation
language yields the largest improvements for individual
languages, underscoring the importance of linguistic
alignment. We also identify specific failure cases where certain
language-quantizer combinations degrade performance, which we trace to differences in activation range distributions across
languages. These results highlight that static one-size-fits-all calibration is suboptimal and that tailoring calibration data, both in
language and diversity, plays a crucial role in robustly quantizing
multilingual LLMs.
Read the full abstract ▾