Computation and Language
Style Transfer as Bias Mitigation: Diffusion Models for Synthetic Mental Health Text for Arabic
Saad Mankarious and Aya Zirikly
Synthetic data offers a promising solution for mitigating data scarcity and demographic bias in mental health analysis, yet existing approaches largely rely on pretrained large
language models (LLMs), which may suffer from limited output diversity and propagate biases inherited from their training data. In this work, we propose a pretraining-free
diffusion-based approach for synthetic text generation that frames bias mitigation as a style transfer problem. Using the CARMA Arabic mental health corpus, which exhibits a substantial gender imbalance, we focus on male-to-female style transfer to augment underrepresented female-authored content. We construct five
datasets capturing varying linguistic and semantic aspects of gender expression in Arabic and train separate
diffusion models for each setting. Quantitative evaluations demonstrate consistently high semantic fidelity between source and generated text, alongside meaningful surface-level stylistic divergence, while qualitative analysis confirms linguistically plausible gender transformations. Our results show that
diffusion-based style transfer can generate high-
entropy, semantically faithful synthetic data without reliance on pretrained LLMs, providing an effective and flexible framework for mitigating gender bias in sensitive, low-resource mental health domains.
Read the full abstract ▾
Computation and Language
A Systematic Analysis of Chunking Strategies for Reliable Question Answering
Sofia Bennani and Charles Moslonka
We study how document chunking choices impact the reliability of
Retrieval-Augmented Generation (RAG) systems in industry. While practice often relies on heuristics, our end-to-end evaluation on Natural Questions systematically varies chunking method (
token, sentence, semantic, code), chunk size, overlap, and context length. We use a standard industrial setup: SPLADE
retrieval and a Mistral-8B generator. We derive actionable lessons for cost-efficient deployment: (i) overlap provides no measurable benefit and increases indexing cost; (ii) sentence chunking is the most cost-effective method, matching semantic chunking up to ~5k
tokens; (iii) a "context cliff" reduces quality beyond ~2.5k
tokens; and (iv) optimal context depends on the goal (semantic quality peaks at small contexts; exact match at larger ones).
Read the full abstract ▾
Computation and Language
NewsRECON: News Article Retrieval for Image Contextualization
Jonathan Tonglet, Iryna Gurevych, Tinne Tuytelaars et al. (+1 more)
Identifying when and where a news image was taken is crucial for journalists and forensic experts to produce credible stories and debunk misinformation. While many existing methods rely on reverse image search (RIS) engines, these tools often fail to return results, thereby limiting their practical applicability. In this work, we address the challenging scenario where RIS evidence is unavailable. We investigate the potential of news article corpora as an alternative to RIS, linking images to relevant articles to infer their dates and locations from article metadata. We evaluate the performance of a news article
retrieval pipeline, NewsRECON, which leverages a corpus of over 85,000 articles. Experiments on the TARA
dataset show that NewsRECON outperforms prior work and can be combined with a
multimodal large
language model (MLLM) to achieve new SOTA results in the absence of RIS evidence. Furthermore, NewsRECON generalizes to the 5Pils-OOC
benchmark despite geographic and temporal shifts. We make our code available.
Read the full abstract ▾
All authors Jonathan Tonglet, Iryna Gurevych, Tinne Tuytelaars, Marie-Francine Moens
Computation and Language
Learning to Explain: Supervised Token Attribution from Transformer Attention Patterns
George Mihaila
Explainable AI (XAI) has become critical as
transformer-based models are deployed in high-stakes applications including healthcare, legal systems, and financial services, where opacity hinders trust and accountability.
Transformers self-
attention mechanisms have proven valuable for model interpretability, with
attention weights successfully used to understand model focus and behavior (Xu et al., 2015); (Wiegreffe and Pinter, 2019). However, existing
attention-based explanation methods rely on manually defined aggregation strategies and fixed attribution rules (Abnar and Zuidema, 2020a); (Chefer et al., 2021), while model-agnostic approaches (LIME, SHAP) treat the model as a black box and incur significant computational costs through input perturbation. We introduce Explanation Network (ExpNet), a lightweight neural network that learns an explicit mapping from
transformer attention patterns to
token-level importance scores. Unlike prior methods, ExpNet discovers optimal
attention feature combinations automatically rather than relying on predetermined rules. We evaluate ExpNet in a challenging cross-task setting and
benchmark it against a broad spectrum of model-agnostic methods and
attention-based techniques spanning four methodological families.
Read the full abstract ▾
Computation and Language
Truth with a Twist: The Rhetoric of Persuasion in Professional vs. Community-Authored Fact-Checks
Olesya Razuvayevskaya and Kalina Bontcheva
This study presents the first large-scale comparison of persuasion techniques present in crowd- versus professionally-written debunks. Using extensive
datasets from Community Notes (CNs), EUvsDisinfo, and the Database of Known Fakes (DBKF), we quantify the prevalence and types of persuasion techniques across these fact-checking ecosystems. Contrary to prior hypothesis that community-produced debunks rely more heavily on subjective or persuasive wording, we find no evidence that CNs contain a higher average number of persuasion techniques than professional fact-checks. We additionally identify systematic rhetorical differences between CNs and professional debunking efforts, reflecting differences in institutional norms and topical coverage. Finally, we examine how the crowd evaluates persuasive
language in CNs and show that, although notes with more persuasive elements receive slightly higher overall helpfulness ratings, crowd raters are effective at penalising the use of particular problematic rhetorical means
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
DermaBench: A Clinician-Annotated Benchmark Dataset for Dermatology Visual Question Answering and Reasoning
Abdurrahim Yilmaz, Ozan Erdem, Ece Gokyayla et al. (+5 more)
Vision-
language models (VLMs) are increasingly important in medical applications; however, their evaluation in dermatology remains limited by
datasets that focus primarily on image-level classification tasks such as lesion recognition. While valuable for recognition, such
datasets cannot assess the full visual understanding,
language grounding, and clinical
reasoning capabilities of
multimodal models. Visual question answering (VQA)
benchmarks are required to evaluate how models interpret dermatological images, reason over fine-grained morphology, and generate clinically meaningful descriptions. We introduce DermaBench, a clinician-annotated dermatology VQA
benchmark built on the Diverse Dermatology Images (DDI)
dataset. DermaBench comprises 656 clinical images from 570 unique patients spanning Fitzpatrick skin types I-VI. Using a hierarchical annotation schema with 22 main questions (single-choice, multi-choice, and open-ended), expert dermatologists annotated each image for diagnosis, anatomic site, lesion morphology, distribution, surface features, color, and image quality, together with open-ended narrative descriptions and summaries, yielding approximately 14.474 VQA-style annotations. DermaBench is released as a metadata-only
dataset to respect upstream licensing and is publicly available at Harvard Dataverse.
Read the full abstract ▾
All authors Abdurrahim Yilmaz, Ozan Erdem, Ece Gokyayla, Ayda Acar, Burc Bugra Dagtas, Dilara Ilhan Erdil, Gulsum Gencoglan, Burak Temelkuran
Computation and Language
XCR-Bench: Benchmarking Cross-Cultural Reasoning in LLMs via Culture-Specific Items and Hall's Triad
Mohsinul Kabir, Tasnim Ahmed, Md Mezbaur Rahman et al. (+5 more)
Cross-cultural competence in large
language models (LLMs) requires understanding and adapting Culture-Specific Items (CSIs) across varying cultural contexts. However, progress in evaluating this capability remains limited by the lack of high-quality CSI-annotated corpora with parallel cross-cultural sentence pairs. We introduce XCR-Bench, a Cross(X)-Cultural
Reasoning Benchmark containing 4.1k parallel sentences and 1,098 CSIs across three
reasoning tasks. XCR-Bench integrates Newmark's CSI framework with
Hall's Triad of Culture, enabling evaluation across levels of cultural visibility – from observable practices to implicit social norms and values. Experiments on eight
multilingual LLMs show that state-of-the-art models exhibit consistent weaknesses in identifying and adapting specific categories of CSIs, revealing a gap between surface-level recall and explicit cultural
reasoning. Performance declines significantly on culturally sensitive categories and deeper cultural levels (p<0.005, 8/8 models), and adaptation quality varies systematically across target cultures and Bengali regional variants, indicating encoded regional and ethno-religious biases even within a single linguistic setting. We publicly release the corpus and code to support future research on cross-cultural NLP.
Read the full abstract ▾
All authors Mohsinul Kabir, Tasnim Ahmed, Md Mezbaur Rahman, Shaoxiong Ji, Hassan Alhuzali, Yuechen Jiang, Jimin Huang, Sophia Ananiadou
Computation and Language
Peter Devine, Mardhiyah Sanni, Farid Adilazuarda et al. (+2 more)
We present Kakugo, a novel and cost-effective pipeline designed to train general-purpose Small
Language Models (SLMs) for low-resource
languages using only the
language name as input. By using a large teacher model to generate synthetic
prompts and translate instruction
datasets, we produced training data and SLMs for 54 low-resource
languages. Evaluations across a diverse set of general natural
language processing tasks, including translation, classification, and question answering, demonstrate that our pipeline consistently improves performance over base models. With a total generation and training cost of under $50 per
language, Kakugo offers an accessible method for communities to develop
language-specific AI.
Read the full abstract ▾
All authors Peter Devine, Mardhiyah Sanni, Farid Adilazuarda, Julieta Gil Loizaga, Barry Haddow
Computation and Language
Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steering
Yuxin Chen, Zhengzhou Cai, Xiangtian Ji et al. (+4 more)
Mixture-of-Experts (MoE) architectures have shown strong
multilingual capabilities, yet the internal mechanisms underlying performance gains and cross-
language differences remain insufficiently understood. In this work, we conduct a systematic analysis of MoE models, examining routing behavior and expert specialization across
languages and network depth. Our analysis reveals that
multilingual processing in MoE models is highly structured: routing aligns with linguistic families, expert utilization follows a clear layerwise pattern, and high-resource
languages rely on shared experts while low-resource
languages depend more on
language-exclusive experts despite weaker performance. Layerwise interventions further show that early and late MoE layers support
language-specific processing, whereas middle layers serve as
language-agnostic capacity hubs. Building on these insights, we propose a routing-guided steering method that adaptively guides routing behavior in middle layers toward shared experts associated with dominant
languages at inference time, leading to consistent
multilingual performance improvements, particularly for linguistically related
language pairs. Our code is available at https://github.com/conctsai/Multilingualism-in-Mixture-of-Experts-LLMs.
Read the full abstract ▾
All authors Yuxin Chen, Zhengzhou Cai, Xiangtian Ji, Weixiang Zhao, An Zhang, Xiang Wang, Tat-Seng Chua
Computation and Language
PRiSM: Benchmarking Phone Realization in Speech Models
Shikhar Bharadwaj, Chin-Jou Li, Yoonjae Kim et al. (+13 more)
Phone recognition (PR) serves as the atomic interface for
language-agnostic modeling for cross-lingual
speech processing and phonetic analysis. Despite prolonged efforts in developing PR systems, current evaluations only measure surface-level transcription accuracy. We introduce PRiSM, the first open-source
benchmark designed to expose blind spots in phonetic perception through intrinsic and extrinsic evaluation of PR systems. PRiSM standardizes transcription-based evaluation and assesses downstream utility in clinical, educational, and
multilingual settings with transcription and representation probes. We find that diverse
language exposure during training is key to PR performance, encoder-CTC models are the most stable, and specialized PR models still outperform Large Audio
Language Models. PRiSM releases code, recipes, and
datasets to move the field toward
multilingual speech models with robust phonetic ability: https://github.com/changelinglab/prism.
Read the full abstract ▾
All authors Shikhar Bharadwaj, Chin-Jou Li, Yoonjae Kim, Kwanghee Choi, Eunjung Yeo, Ryan Soh-Eun Shim, Hanyu Zhou, Brendon Boldt, Karen Rosero Jacome, Kalvin Chang, Darsh Agrawal, Keer Xu, Chao-Han Huck Yang, Jian Zhu, Shinji Watanabe, David R. Mortensen
Computation and Language
Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variants
Yunhe Wang, Kai Han, Huiling Zhen et al. (+13 more)
The paradigm of Large
Language Models (LLMs) is currently defined by auto-regressive (AR) architectures, which generate text through a sequential “brick-by-brick” process. Despite their success, AR models are inherently constrained by a causal bottleneck that limits global structural foresight and iterative refinement.
Diffusion Language Models (DLMs) offer a transformative alternative, conceptualizing text generation as a holistic, bidirectional denoising process akin to a sculptor refining a masterpiece. However, the potential of DLMs remains largely untapped as they are frequently confined within AR-legacy infrastructures and optimization frameworks. In this Perspective, we identify ten fundamental challenges ranging from architectural inertia and gradient sparsity to the limitations of linear
reasoning that prevent DLMs from reaching their “GPT-4 moment”. We propose a strategic roadmap organized into four pillars: foundational infrastructure, algorithmic optimization, cognitive
reasoning, and unified
multimodal intelligence. By shifting toward a
diffusion-native ecosystem characterized by multi-scale tokenization, active remasking, and latent thinking, we can move beyond the constraints of the causal horizon. We argue that this transition is essential for developing next-generation AI capable of complex structural
reasoning, dynamic self-correction, and seamless
multimodal integration.
Read the full abstract ▾
All authors Yunhe Wang, Kai Han, Huiling Zhen, Yuchuan Tian, Hanting Chen, Yongbing Huang, Yufei Cui, Yingte Shu, Shan Gao, Ismail Elezi, Roy Vaughan Miles, Songcen Xu, Feng Wen, Chao Xu, Sinan Zeng, Dacheng Tao
Computation and Language
RM-Distiller: Exploiting Generative LLM for Reward Model Distillation
Hongli Zhou, Hui Huang, Wei Liu et al. (+8 more)
Reward models (RMs) play a pivotal role in aligning large
language models (LLMs) with human preferences. Due to the difficulty of obtaining high-quality human preference annotations, distilling preferences from generative LLMs has emerged as a standard practice. However, existing approaches predominantly treat teacher models as simple binary annotators, failing to fully exploit the rich knowledge and capabilities for RM
distillation. To address this, we propose RM-Distiller, a framework designed to systematically exploit the multifaceted capabilities of teacher LLMs: (1) Refinement capability, which synthesizes highly correlated response pairs to create fine-grained and contrastive signals. (2) Scoring capability, which guides the RM in capturing precise preference strength via a margin-aware optimization objective. (3) Generation capability, which incorporates the teacher's generative distribution to regularize the RM to preserve its fundamental linguistic knowledge. Extensive experiments demonstrate that RM-Distiller significantly outperforms traditional
distillation methods both on RM
benchmarks and reinforcement learning-based
alignment, proving that exploiting multifaceted teacher capabilities is critical for effective reward modeling. To the best of our knowledge, this is the first systematic research on RM
distillation from generative LLMs.
Read the full abstract ▾
All authors Hongli Zhou, Hui Huang, Wei Liu, Chenglong Wang, Xingyuan Bu, Lvyuan Han, Fuhai Song, Muyun Yang, Wenhao Jiang, Hailong Cao, Tiejun Zhao
Computation and Language
BACH-V: Bridging Abstract and Concrete Human-Values in Large Language Models
Junyu Zhang, Yipeng Kang, Jiong Guo et al. (+2 more)
Do large
language models (LLMs) genuinely understand abstract concepts, or merely manipulate them as statistical patterns? We introduce an abstraction-grounding framework that decomposes conceptual understanding into three capacities: interpretation of abstract concepts (Abstract-Abstract, A-A), grounding of abstractions in concrete events (Abstract-Concrete, A-C), and application of abstract principles to regulate concrete decisions (Concrete-Concrete, C-C). Using human values as a testbed - given their semantic richness and centrality to
alignment - we employ probing (detecting value traces in internal activations) and steering (modifying representations to shift behavior). Across six open-source LLMs and ten value dimensions, probing shows that diagnostic probes trained solely on abstract value descriptions reliably detect the same values in concrete event narratives and decision
reasoning, demonstrating cross-level transfer. Steering reveals an asymmetry: intervening on value representations causally shifts concrete judgments and decisions (A-C, C-C), yet leaves abstract interpretations unchanged (A-A), suggesting that encoded abstract values function as stable anchors rather than malleable activations. These findings indicate LLMs maintain structured value representations that bridge abstraction and action, providing a mechanistic and operational foundation for building value-driven autonomous AI systems with more transparent, generalizable
alignment and control.
Read the full abstract ▾
All authors Junyu Zhang, Yipeng Kang, Jiong Guo, Jiayu Zhan, Junqi Wang
Computation and Language
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models
Hengyuan Zhang, Zhihao Zhang, Mingyang Wang et al. (+26 more)
Mechanistic Interpretability (MI) has emerged as a vital approach to demystify the opaque decision-making of Large
Language Models (LLMs). However, existing reviews primarily treat MI as an observational science, summarizing analytical insights while lacking a systematic framework for actionable intervention. To bridge this gap, we present a practical survey structured around the pipeline: "Locate, Steer, and Improve." We formally categorize Localizing (diagnosis) and Steering (intervention) methods based on specific Interpretable Objects to establish a rigorous intervention protocol. Furthermore, we demonstrate how this framework enables tangible improvements in
Alignment, Capability, and Efficiency, effectively operationalizing MI as an actionable methodology for model optimization. The curated paper list of this work is available at https://github.com/rattlesnakey/Awesome-Actionable-MI-Survey.
Read the full abstract ▾
All authors Hengyuan Zhang, Zhihao Zhang, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Ercong Nie, Xufeng Duan, Feijiang Han, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schütze, Xuanjing Huang, Qi Zhang, Ngai Wong
Computation and Language
From Tags to Trees: Structuring Fine-Grained Knowledge for Controllable Data Selection in LLM Instruction Tuning
Zihan Niu, Wenping Hu, Junmin Chen et al. (+3 more)
Effective and controllable data selection is critical for LLM instruction tuning, especially with massive open-source
datasets. Existing approaches primarily rely on instance-level quality scores, or diversity metrics based on
embedding clusters or semantic tags. However, constrained by the flatness of
embedding spaces or the coarseness of tags, these approaches overlook fine-grained knowledge and its intrinsic hierarchical dependencies, consequently hindering precise data valuation and knowledge-aligned sampling. To address this challenge, we propose Tree-aware Aligned Global Sampling (TAGS), a unified framework that leverages a knowledge tree built from fine-grained tags, thereby enabling joint control of global quality, diversity, and target
alignment. Using an LLM-based tagger, we extract atomic knowledge concepts, which are organized into a global tree through bottom-up hierarchical clustering. By grounding data instances onto this tree, a tree-aware metric then quantifies data quality and diversity, facilitating effective sampling. Our controllable sampling strategy maximizes tree-level information gain and enforces leaf-level
alignment via KL-divergence for specific domains. Extensive experiments demonstrate that TAGS significantly outperforms state-of-the-art baselines. Notably, it surpasses the full-
dataset model by +5.84% using only 5% of the data, while our aligned sampling strategy further boosts average performance by +4.24%.
Read the full abstract ▾
All authors Zihan Niu, Wenping Hu, Junmin Chen, Xiyue Wang, Tong Xu, Ruiming Tang
Computation and Language
"The Whole Is Greater Than the Sum of Its Parts": A Compatibility-Aware Multi-Teacher CoT Distillation Framework
Jin Cui, Jiaqi Guo, Ruixuan Yang et al. (+6 more)
Chain-of-Thought (CoT)
reasoning empowers Large
Language Models (LLMs) with remarkable capabilities but typically requires prohibitive parameter scales. CoT
distillation has emerged as a promising paradigm to transfer
reasoning prowess into compact Student Models (SLMs), but existing approaches often rely on a solitary teacher, capping the student's potential since individual LLMs often exhibit distinct capability biases and may suffer from catastrophic forgetting. While leveraging diverse teachers seems appealing, effectively fusing their supervisions remains challenging: teacher-student incompatibility risks amplifying
hallucinations, and passive supervision fails to ensure genuine logic internalization. To address this, we introduce COMPACT, a framework that adaptively fuses supervisions from different teachers by dynamically weighting teacher gradients based on the student's real-time compatibility evaluated by a multi-dimensional metric: (1) Graph-based Consensus to filter misleading rationales by identifying mainstream
reasoning paths; (2) Mutual-Information-based Adaptability to detect "epiphany moments" for genuinely understanding the
reasoning process rather than merely imitating; and (3) Loss-based Difficulty to assess student receptivity to the teacher's guidance and prevent negative transfer. Extensive experiments and latent space analysis demonstrate that COMPACT effectively integrates diverse
reasoning capabilities without damaging the model's original knowledge structure, achieving state-of-the-art performance on various
benchmarks while mitigating catastrophic forgetting.
Read the full abstract ▾
All authors Jin Cui, Jiaqi Guo, Ruixuan Yang, Jiayi Lu, Jiepeng Zhou, Jiajun Xu, Jiangcheng Song, Boran Zhao, Pengju Ren
Computation and Language · Cross-listed from physics.soc-ph
Generating consensus and dissent on massive discussion platforms with a semantic-vector model
A. Ferrer, D. Muñoz-Jordán, A. Rivero et al. (+3 more)
Reaching consensus on massive discussion networks is critical for reducing noise and achieving optimal collective outcomes. However, the natural tendency of humans to preserve their initial ideas constrains the emergence of global solutions. To address this, Collective Intelligence (CI) platforms facilitate the discovery of globally superior solutions. We introduce a dynamical system based on the standard $O(N)$ model to drive the aggregation of semantically similar ideas. The system consists of users represented as nodes in a $d=2$
lattice with nearest-neighbor interactions, where their ideas are represented by semantic vectors computed with a pretrained
embedding model. We analyze the system's
equilibrium states as a function of the coupling parameter $\beta$. Our results show that $\beta > 0$ drives the system toward a
ferromagnetic-like phase (global consensus), while $\beta < 0$ induces an
antiferromagnetic-like state (maximum dissent), where users maximize semantic distance from their neighbors. This framework offers a controllable method for managing the tradeoff between cohesion and diversity in CI platforms.
Read the full abstract ▾
All authors A. Ferrer, D. Muñoz-Jordán, A. Rivero, A. Tarancón, C. Tarancón, D. Yllanes
Computation and Language
Automatic Prompt Optimization for Dataset-Level Feature Discovery
Adrian Cosma, Oleg Szehr, David Kletz et al. (+2 more)
Feature extraction from unstructured text is a critical step in many downstream classification pipelines, yet current approaches largely rely on hand-crafted
prompts or fixed feature schemas. We formulate feature discovery as a
dataset-level
prompt optimization problem: given a labelled text corpus, the goal is to induce a global set of interpretable and discriminative feature definitions whose realizations optimize a downstream supervised learning objective. To this end, we propose a multi-
agent prompt optimization framework in which
language-model
agents jointly propose feature definitions, extract feature values, and evaluate feature quality using
dataset-level performance and interpretability feedback. Instruction
prompts are iteratively refined based on this structured feedback, enabling optimization over
prompts that induce shared feature sets rather than per-example predictions. This formulation departs from prior
prompt optimization methods that rely on per-sample supervision and provides a principled mechanism for automatic feature discovery from unstructured text.
Read the full abstract ▾
All authors Adrian Cosma, Oleg Szehr, David Kletz, Alessandro Antonucci, Olivier Pelletier
Computation and Language
HyperWalker: Dynamic Hypergraph-Based Deep Diagnosis for Multi-Hop Clinical Modeling across EHR and X-Ray in Medical VLMs
Yuezhe Yang, Hao Wang, Yige Peng et al. (+2 more)
Automated clinical diagnosis remains a core challenge in medical AI, which usually requires models to integrate multi-modal data and reason across complex, case-specific contexts. Although recent methods have advanced medical report generation (MRG) and visual question answering (VQA) with medical
vision-
language models (VLMs), these methods, however, predominantly operate under a sample-isolated inference paradigm, as such processing cases independently without access to longitudinal electronic health records (EHRs) or structurally related patient examples. This paradigm limits
reasoning to image-derived information alone, which ignores external complementary medical evidence for potentially more accurate diagnosis. To overcome this limitation, we propose HyperWalker, a Deep Diagnosis framework that reformulates clinical
reasoning via dynamic hypergraphs and test-time training. First, we construct a dynamic hypergraph, termed iBrochure, to model the structural heterogeneity of EHR data and implicit high-order associations among
multimodal clinical information. Within this hypergraph, a reinforcement learning
agent, Walker, navigates to and identifies optimal diagnostic paths. To ensure comprehensive coverage of diverse clinical characteristics in test samples, we incorporate a linger mechanism, a multi-hop orthogonal
retrieval strategy that iteratively selects clinically complementary neighborhood cases reflecting distinct clinical attributes. Experiments on MRG with MIMIC and medical VQA on EHRXQA demonstrate that HyperWalker achieves state-of-the-art performance. Code is available at: https://github.com/Bean-Young/HyperWalker
Read the full abstract ▾
All authors Yuezhe Yang, Hao Wang, Yige Peng, Jinman Kim, Lei Bi
Computation and Language
AgentEHR: Advancing Autonomous Clinical Decision-Making via Retrospective Summarization
Yusheng Liao, Chuan Xuan, Yutong Cai et al. (+4 more)
Large
Language Models have demonstrated profound utility in the medical domain. However, their application to autonomous Electronic Health Records~(EHRs)
navigation remains constrained by a reliance on curated inputs and simplified
retrieval tasks. To bridge the gap between idealized experimental settings and realistic clinical environments, we present AgentEHR. This
benchmark challenges
agents to execute complex decision-making tasks, such as diagnosis and treatment planning, requiring long-range interactive
reasoning directly within raw and high-noise databases. In tackling these tasks, we identify that existing summarization methods inevitably suffer from critical information loss and fractured
reasoning continuity. To address this, we propose RetroSum, a novel framework that unifies a retrospective summarization mechanism with an evolving experience strategy. By dynamically re-evaluating interaction history, the retrospective mechanism prevents long-context information loss and ensures unbroken logical coherence. Additionally, the evolving strategy bridges the domain gap by retrieving accumulated experience from a
memory bank. Extensive empirical evaluations demonstrate that RetroSum achieves performance gains of up to 29.16% over competitive baselines, while significantly decreasing total interaction errors by up to 92.3%.
Read the full abstract ▾
All authors Yusheng Liao, Chuan Xuan, Yutong Cai, Lina Yang, Zhe Chen, Yanfeng Wang, Yu Wang
Computation and Language
Confident Rankings with Fewer Items: Adaptive LLM Evaluation with Continuous Scores
Esma Balk{\i}r, Alice Pernthaller, Marco Basaldella et al. (+2 more)
Computerized Adaptive Testing (CAT) has proven effective for efficient LLM evaluation on multiple-choice
benchmarks, but modern LLM evaluation increasingly relies on generation tasks where outputs are scored continuously rather than marked correct/incorrect. We present a principled extension of IRT-based adaptive testing to continuous bounded scores (ROUGE, BLEU, LLM-as-a-Judge) by replacing the Bernoulli response distribution with a heteroskedastic normal distribution. Building on this, we introduce an uncertainty aware ranker with adaptive stopping criteria that achieves reliable model ranking while testing as few items and as cheaply as possible. We validate our method on five
benchmarks spanning n-gram-based,
embedding-based, and LLM-as-judge metrics. Our method improves ranking correlation by 0.13 $\tau$ over random sampling and has 99% accuracy on confident predictions while using 2% of the items after a one-time calibration step.
Read the full abstract ▾
All authors Esma Balk{\i}r, Alice Pernthaller, Marco Basaldella, José Hernández-Orallo, Nigel Collier
Computation and Language
OpenLearnLM Benchmark: A Unified Framework for Evaluating Knowledge, Skill, and Attitude in Educational Large Language Models
Unggi Lee, Sookbun Lee, Heungsoo Choi et al. (+11 more)
Large
Language Models are increasingly deployed as educational tools, yet existing
benchmarks focus on narrow skills and lack grounding in learning sciences. We introduce OpenLearnLM
Benchmark, a theory-grounded framework evaluating LLMs across three dimensions derived from educational assessment theory: Knowledge (curriculum-aligned content and pedagogical understanding), Skills (scenario-based competencies organized through a four-level center-role-scenario-subscenario hierarchy), and Attitude (
alignment consistency and deception resistance). Our
benchmark comprises 124K+ items spanning multiple subjects, educational roles, and difficulty levels based on Bloom's taxonomy. The Knowledge domain prioritizes authentic assessment items from established
benchmarks, while the Attitude domain adapts Anthropic's
Alignment Faking methodology to detect behavioral inconsistency under varying monitoring conditions. Evaluation of seven frontier models reveals distinct capability profiles: Claude-Opus-4.5 excels in practical skills despite lower content knowledge, while Grok-4.1-fast leads in knowledge but shows
alignment concerns. Notably, no single model dominates all dimensions, validating the necessity of multi-axis evaluation. OpenLearnLM provides an open, comprehensive framework for advancing LLM readiness in authentic educational contexts.
Read the full abstract ▾
All authors Unggi Lee, Sookbun Lee, Heungsoo Choi, Jinseo Lee, Haeun Park, Younghoon Jeon, Sungmin Cho, Minju Kang, Junbo Koh, Jiyeong Bae, Minwoo Nam, Juyeon Eun, Yeonji Jung, Yeil Jeong
Computation and Language · Cross-listed from cs.MM
Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring
Dongxu Zhang, Yiding Sun, Cheng Tan et al. (+4 more)
While Chain-of-Thought (CoT)
reasoning significantly enhances the performance of
Multimodal Large
Language Models (MLLMs), its autoregressive nature incurs prohibitive latency constraints. Current efforts to mitigate this via
token compression often fail by blindly applying text-centric metrics to
multimodal contexts. We identify a critical failure mode termed Visual Amnesia, where linguistically redundant
tokens are erroneously pruned, leading to
hallucinations. To address this, we introduce V-Skip that reformulates
token pruning as a Visual-Anchored Information Bottleneck (VA-IB) optimization problem. V-Skip employs a dual-path gating mechanism that weighs
token importance through both linguistic surprisal and cross-modal
attention flow, effectively rescuing visually salient anchors. Extensive experiments on Qwen2-VL and Llama-3.2 families demonstrate that V-Skip achieves a $2.9\times$ speedup with negligible accuracy loss. Specifically, it preserves fine-grained visual details, outperforming other baselines over 30% on the DocVQA.
Read the full abstract ▾
All authors Dongxu Zhang, Yiding Sun, Cheng Tan, Wenbiao Yan, Ning Yang, Jihua Zhu, Haijun Zhang
Computation and Language
Pedagogical Alignment for Vision-Language-Action Models: A Comprehensive Framework for Data, Architecture, and Evaluation in Education
Unggi Lee, Jahyun Jeong, Sunyoung Shin et al. (+12 more)
Science demonstrations are important for effective STEM education, yet teachers face challenges in conducting them safely and consistently across multiple occasions, where robotics can be helpful. However, current
Vision-
Language-Action (VLA) models require substantial computational resources and sacrifice
language generation capabilities to maximize efficiency, making them unsuitable for resource-constrained educational settings that require interpretable, explanation-generating systems. We present Pedagogical VLA Framework, a framework that applies pedagogical
alignment to lightweight VLA models through four components: text healing to restore
language generation capabilities, large
language model (LLM)
distillation to transfer pedagogical knowledge, safety training for educational environments, and pedagogical evaluation adjusted to science education contexts. We evaluate Pedagogical VLA Framework across five science demonstrations spanning physics, chemistry, biology, and earth science, using an evaluation framework developed in collaboration with science education experts. Our evaluation assesses both task performance (success rate, protocol compliance, efficiency, safety) and pedagogical quality through teacher surveys and LLM-as-Judge assessment. We additionally provide qualitative analysis of generated texts. Experimental results demonstrate that Pedagogical VLA Framework achieves comparable task performance to baseline models while producing contextually appropriate educational explanations.
Read the full abstract ▾
All authors Unggi Lee, Jahyun Jeong, Sunyoung Shin, Haeun Park, Jeongsu Moon, Youngchang Song, Jaechang Shim, JaeHwan Lee, Yunju Noh, Seungwon Choi, Ahhyun Kim, TaeHyeon Kim, Kyungtae Joo, Taeyeong Kim, Gyeonggeon Lee
Computation and Language
FutureOmni: Evaluating Future Forecasting from Omni-Modal Context for Multimodal LLMs
Qian Chen, Jinlan Fu, Changsong Li et al. (+3 more)
Although
Multimodal Large
Language Models (MLLMs) demonstrate strong omni-modal perception, their ability to forecast future events from audio-visual cues remains largely unexplored, as existing
benchmarks focus mainly on retrospective understanding. To bridge this gap, we introduce FutureOmni, the first
benchmark designed to evaluate omni-modal future
forecasting from audio-visual environments. The evaluated models are required to perform cross-modal causal and temporal
reasoning, as well as effectively leverage internal knowledge to predict future events. FutureOmni is constructed via a scalable LLM-assisted, human-in-the-loop pipeline and contains 919
videos and 1,034 multiple-choice QA pairs across 8 primary domains. Evaluations on 13 omni-modal and 7
video-only models show that current systems struggle with audio-visual future prediction, particularly in
speech-heavy scenarios, with the best accuracy of 64.8% achieved by Gemini 3 Flash. To mitigate this limitation, we curate a 7K-sample instruction-tuning
dataset and propose an Omni-Modal Future
Forecasting (OFF) training strategy. Evaluations on FutureOmni and popular audio-visual and
video-only
benchmarks demonstrate that OFF enhances future
forecasting and generalization. We publicly release all code (https://github.com/OpenMOSS/FutureOmni) and
datasets (https://huggingface.co/
datasets/OpenMOSS-Team/FutureOmni).
Read the full abstract ▾
All authors Qian Chen, Jinlan Fu, Changsong Li, Min Zhang, See-Kiong Ng, Xipeng Qiu
Computation and Language
The Role of Prosodic and Lexical Cues in Turn-Taking with Self-Supervised Speech Representations
Sam OConnor Russell, Delphine Charuau and Naomi Harte
Fluid turn-taking remains a key challenge in human-
robot interaction. Self-supervised
speech representations (S3Rs) have driven many advances, but it remains unclear whether S3R-based turn-taking models rely on prosodic cues, lexical cues or both. We introduce a vocoder-based approach to control prosody and lexical cues in
speech more cleanly than prior work. This allows us to probe the voice-activity projection model, an S3R-based turn-taking model. We find that prediction on prosody-matched, unintelligible noise is similar to accuracy on clean
speech. This reveals both prosodic and lexical cues support turn-taking, but either can be used in isolation. Hence, future models may only require prosody, providing
privacy and potential performance benefits. When either prosodic or lexical information is disrupted, the model exploits the other without further training, indicating they are encoded in S3Rs with limited interdependence. Results are consistent in CPC-based and wav2vec2.0 S3Rs. We discuss our findings and highlight a number of directions for future work. All code is available to support future research.
Read the full abstract ▾
Computation and Language
Knowledge Graph-Assisted LLM Post-Training for Enhanced Legal Reasoning
Dezhao Song, Guglielmo Bonifazi, Frank Schilder et al. (+1 more)
LLM post-training has primarily relied on large text corpora and human feedback, without capturing the structure of domain knowledge. This has caused models to struggle dealing with complex
reasoning tasks, especially for high-stakes professional domains. In Law,
reasoning requires deep understanding of the relations between various legal concepts, a key component missing in current LLM post-training. In this paper, we propose a knowledge graph (KG)-assisted approach for enhancing LLMs'
reasoning capability in Legal that is generalizable to other high-stakes domains. We model key legal concepts by following the IRAC (Issue, Rule, Analysis and Conclusion) framework, and construct a KG with 12K legal cases. We then produce training data using our IRAC KG, and conduct both Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) with three state-of-the-art (SOTA) LLMs (30B, 49B and 70B), varying architecture and base model family. Our post-trained models obtained better average performance on 4/5 diverse legal
benchmarks (14 tasks) than baselines. In particular, our 70B DPO model achieved the best score on 4/6
reasoning tasks, among baselines and a 141B SOTA legal LLM, demonstrating the effectiveness of our KG for enhancing LLMs' legal
reasoning capability.
Read the full abstract ▾
All authors Dezhao Song, Guglielmo Bonifazi, Frank Schilder, Jonathan Richard Schwarz
Computation and Language
Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis
Yushen Chen, Junzhe Liu, Yujie Tu et al. (+6 more)
Arabic spans over 30 spoken varieties, yet no open-source text-to-
speech system unifies them. Key barriers include substantial cross-dialect lexical and phonological divergence, scarce synthesis-grade data, and the absence of a standardized multi-dialect evaluation
benchmark. We present Habibi, a unified-dialectal Arabic TTS framework that addresses all three. Through a multi-step curation pipeline, we repurpose open-source ASR corpora into TTS training data covering 12+ regional dialects. A linguistically-informed curriculum learning strategy - progressing from Modern Standard Arabic to dialectal data - enables robust zero-shot synthesis without text diacritization. We further release the first standardized multi-dialect Arabic TTS
benchmark, comprising over 11,000 utterances across 7 dialect subsets with manually verified transcripts. On this
benchmark, our unified model matches or surpasses per-dialect specialized models. Both automatic metrics and human evaluations confirm that Habibi is highly competitive with ElevenLabs' Eleven v3 (alpha) in intelligibility, speaker similarity, and naturalness. Extensive ablations (~8,000 H100 GPU hours, 30+ configurations) validate each design choice. We open-source all checkpoints, training and inference code, and
benchmark data - the first such release for multi-dialect Arabic TTS - at https://SWivid.github.io/Habibi/ .
Read the full abstract ▾
All authors Yushen Chen, Junzhe Liu, Yujie Tu, Zhikang Niu, Yuzhe Liang, Chunyu Qiang, Chen Zhang, Kai Yu, Xie Chen
Computation and Language · Cross-listed from cs.AI
Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance
Mostapha Benhenda
We introduce Look-Ahead-Bench, a standardized
benchmark measuring look-ahead bias in Point-in-Time (PiT) Large
Language Models (LLMs) within realistic and practical financial workflows. Unlike most existing approaches that primarily test inner lookahead knowledge via Q\&A, our
benchmark evaluates model behavior in practical scenarios. To distinguish genuine predictive capability from memorization-based performance, we analyze performance decay across temporally distinct
market regimes, incorporating several quantitative baselines to establish performance thresholds. We evaluate prominent open-source LLMs – Llama 3.1 (8B and 70B) and DeepSeek 3.2 – against a family of Point-in-Time LLMs (Pitinf-Small, Pitinf-Medium, and frontier-level model Pitinf-Large) from PiT-Inference. Results reveal significant lookahead bias in standard LLMs, as measured with alpha decay, unlike Pitinf models, which demonstrate improved generalization and
reasoning abilities as they scale in size. This work establishes a foundation for the standardized evaluation of temporal bias in financial LLMs and provides a practical framework for identifying models suitable for real-world deployment. Code is available on GitHub: https://github.com/benstaf/lookaheadbench
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution
Shengda Fan, Xuyan Ye and Yankai Lin
Self-play with large
language models has emerged as a promising paradigm for achieving self-improving artificial intelligence. However, existing self-play frameworks often suffer from optimization instability, due to (i) non-stationary objectives induced by solver-dependent reward feedback for the Questioner, and (ii) bootstrapping errors from self-generated pseudo-labels used to supervise the Solver. To mitigate these challenges, we introduce DARC (Decoupled Asymmetric
Reasoning Curriculum), a two-stage framework that stabilizes the self-
evolution process. First, we train the Questioner to synthesize difficulty-calibrated questions, conditioned on explicit difficulty levels and external corpora. Second, we train the Solver with an asymmetric self-
distillation mechanism, where a document-augmented teacher generates high-quality pseudo-labels to supervise the student Solver that lacks document access. Empirical results demonstrate that DARC is model-agnostic, yielding an average improvement of 10.9 points across nine
reasoning benchmarks and three backbone models. Moreover, DARC consistently outperforms all baselines and approaches the performance of fully supervised models without relying on human annotations. The code is available at https://github.com/RUCBM/DARC.
Read the full abstract ▾