Computation and Language
How Pragmatics Shape Articulation: A Computational Case Study in STEM ASL Discourse
Saki Imai, Lee Kezar, Laurel Aichler et al. (+5 more)
Most state-of-the-art sign
language models are trained on interpreter or isolated vocabulary data, which overlooks the variability that characterizes natural dialogue. However, human communication dynamically adapts to contexts and interlocutors through spatiotemporal changes and articulation style. This specifically manifests itself in educational settings, where novel vocabularies are used by teachers, and students. To address this gap, we collect a motion capture
dataset of American Sign
Language (ASL) STEM (Science, Technology, Engineering, and Mathematics) dialogue that enables quantitative comparison between dyadic interactive signing, solo signed lecture, and interpreted articles. Using continuous kinematic features, we disentangle dialogue-specific entrainment from individual effort reduction and show spatiotemporal changes across repeated mentions of STEM terms. On average, dialogue signs are 24.6%-44.6% shorter in duration than the isolated signs, and show significant reductions absent in monologue contexts. Finally, we evaluate sign
embedding models on their ability to recognize STEM signs and approximate how entrained the participants become over time. Our study bridges linguistic analysis and computational modeling to understand how pragmatics shape sign articulation and its representation in sign
language technologies.
Read the full abstract ▾
All authors Saki Imai, Lee Kezar, Laurel Aichler, Mert Inan, Erin Walker, Alicia Wooten, Lorna Quandt, Malihe Alikhani
Computation and Language
Beyond Understanding: Evaluating the Pragmatic Gap in LLMs' Cultural Processing of Figurative Language
Mena Attia, Aashiq Muhamed, Mai Alkhamissi et al. (+2 more)
We present a comprehensive evaluation of the ability of large
language models (LLMs) to process culturally grounded
language, specifically to understand and pragmatically use figurative expressions that encode local knowledge and cultural nuance. Using figurative
language as a proxy for cultural nuance and local knowledge, we design evaluation tasks for contextual understanding, pragmatic use, and connotation interpretation in Arabic and English. We evaluate 22 open- and closed-source LLMs on Egyptian Arabic idioms, multidialectal Arabic proverbs, and English proverbs. Our results show a consistent hierarchy: the average accuracy for Arabic proverbs is 4.29% lower than for English proverbs, and performance for Egyptian idioms is 10.28% lower than for Arabic proverbs. For the pragmatic use task, accuracy drops by 14.07% relative to understanding, though providing contextual idiomatic sentences improves accuracy by 10.66%. Models also struggle with connotative meaning, reaching at most 85.58% agreement with human annotators on idioms with 100% inter-annotator agreement. These findings demonstrate that figurative
language serves as an effective diagnostic for cultural
reasoning: while LLMs can often interpret figurative meaning, they face challenges in using it appropriately. To support future research, we release Kinayat, the first
dataset of Egyptian Arabic idioms designed for both figurative understanding and pragmatic use evaluation.
Read the full abstract ▾
All authors Mena Attia, Aashiq Muhamed, Mai Alkhamissi, Thamar Solorio, Mona Diab
Computation and Language
BitSkip: An Empirical Analysis of Quantization and Early Exit Composition in Transformers
Ramshankar Bhuvaneswaran and Handan Liu
The pursuit of efficient Large
Language Models (LLMs) has led to increasingly complex techniques like extreme quantization and dynamic routing. While individual benefits of these methods are well-documented, their compositional effects remain poorly understood. This paper introduces BitSkip, a hybrid architectural framework for systematically exploring these interactions. Counter-intuitively, our findings reveal that a simple 8-bit quantized model without Hadamard transform (BitSkip-V1) not only outperforms its more complex 4-bit and Hadamard-enhanced counterparts but also competes the full-precision baseline in quality (perplexity of 1.13 vs 1.19) . The introduction of Hadamard transforms, even at 8-bit precision, catastrophically degraded performance by over 37,000%, tracing fundamental training instability. Our BitSkip-V1 recipe demonstrates superior early-exit characteristics, with layer 18 providing optimal 32.5% speed gain for minimal 4% quality loss.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.RO
RoboOmni: Proactive Robot Manipulation in Omni-modal Context
Siyin Wang, Jinlan Fu, Feihong Liu et al. (+11 more)
Recent advances in
Multimodal Large
Language Models (MLLMs) have driven rapid progress in
Vision-
Language-Action (VLA) models for robotic manipulation. Although effective in many scenarios, current approaches largely rely on explicit instructions, whereas in real-world interactions, humans rarely issue instructions directly. Effective collaboration requires
robots to infer user intentions proactively. In this work, we introduce cross-modal contextual instructions, a new setting where intent is derived from spoken dialogue, environmental sounds, and visual cues rather than explicit commands. To address this new setting, we present RoboOmni, a Perceiver-Thinker-Talker-Executor framework based on end-to-end omni-modal LLMs that unifies intention recognition, interaction confirmation, and action execution. RoboOmni fuses auditory and visual signals spatiotemporally for robust intention recognition, while supporting direct
speech interaction. To address the absence of training data for proactive intention recognition in robotic manipulation, we build OmniAction, comprising 140k episodes, 5k+ speakers, 2.4k event sounds, 640 backgrounds, and six contextual instruction types. Experiments in simulation and real-world settings show that RoboOmni surpasses text- and ASR-based baselines in success rate, inference speed, intention recognition, and proactive assistance.
Read the full abstract ▾
All authors Siyin Wang, Jinlan Fu, Feihong Liu, Xinzhe He, Huangxuan Wu, Junhao Shi, Kexin Huang, Zhaoye Fei, Jingjing Gong, Zuxuan Wu, Yu-Gang Jiang, See-Kiong Ng, Tat-Seng Chua, Xipeng Qiu
Computation and Language
Evaluating Long-Term Memory for Long-Context Question Answering
Alessandra Terranova, Björn Ross and Alexandra Birch
In order for large
language models to achieve true conversational continuity and benefit from experiential learning, they need
memory. While research has focused on the development of complex
memory systems, it remains unclear which types of
memory are most effective for long-context conversational tasks. We present a systematic evaluation of
memory-augmented methods on long-context dialogues annotated for question-answering tasks that require diverse
reasoning strategies. We analyse full-context prompting, semantic
memory through
retrieval-augmented generation and agentic
memory, episodic
memory through in-context learning, and procedural
memory through
prompt optimization. Our findings show that
memory-augmented approaches reduce
token usage by over 90% while maintaining competitive accuracy.
Memory architecture complexity should scale with model capability, with foundation models benefitting most from RAG, and stronger instruction-tuned models gaining from episodic learning through reflections and more complex agentic semantic
memory. In particular, episodic
memory can help LLMs recognise the limits of their own knowledge.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
MUStReason: A Benchmark for Diagnosing Pragmatic Reasoning in Video-LMs for Multimodal Sarcasm Detection
Anisha Saha, Varsha Suresh, Timothy Hospedales et al. (+1 more)
Sarcasm is a specific type of irony which involves discerning what is said from what is meant. Detecting sarcasm depends not only on the literal content of an utterance but also on non-verbal cues such as speaker's tonality, facial expressions and conversational context. However, current
multimodal models struggle with complex tasks like sarcasm detection, which require identifying relevant cues across modalities and pragmatically
reasoning over them to infer the speaker's intention. To explore these limitations in VideoLMs, we introduce MUStReason, a diagnostic
benchmark enriched with annotations of modality-specific relevant cues and underlying
reasoning steps to identify sarcastic intent. In addition to benchmarking sarcasm classification performance in VideoLMs, using MUStReason we quantitatively and qualitatively evaluate the generated
reasoning by disentangling the problem into perception and
reasoning, we propose PragCoT, a framework that steers VideoLMs to focus on implied intentions over literal meaning, a property core to detecting sarcasm.
Read the full abstract ▾
All authors Anisha Saha, Varsha Suresh, Timothy Hospedales, Vera Demberg
Computation and Language · Cross-listed from cs.LG
Variational Masked Diffusion Models
Yichi Zhang, Alex Schwing and Zhizhen Zhao
Masked
diffusion models have recently emerged as a flexible framework for discrete generative modeling. However, a key limitation of standard masked
diffusion is its inability to effectively capture dependencies among
tokens that are predicted concurrently, leading to degraded generation quality when dependencies among
tokens are important. To explicitly model dependencies among
tokens, we propose Variational Masked
Diffusion (VMD), a framework that introduces latent variables into the masked
diffusion process. Through controlled experiments on synthetic
datasets, we demonstrate that VMD successfully learns dependencies that conventional masked
diffusion fails to capture. We further validate the effectiveness of our approach on Sudoku puzzles and text
datasets, where learning of dependencies among
tokens improves global consistency. Across these domains, VMD enhances both generation quality and dependency awareness, highlighting the value of integrating variational inference into masked
diffusion. Our code is available at: https://riccizz.github.io/VMD.
Read the full abstract ▾
Computation and Language
Think Twice: Branch-and-Rethink Reasoning Reward Model
Yizhu Jiao, Jiaqi Zeng, Julien Veron Vialard et al. (+3 more)
Large
language models (LLMs) increasingly rely on thinking models that externalize intermediate steps and allocate extra test-time compute, with think-twice strategies showing that a deliberate second pass can elicit stronger
reasoning. In contrast, most reward models (RMs) still compress many quality dimensions into a single scalar in one shot, a design that induces judgment
diffusion:
attention spreads across evaluation criteria, yielding diluted focus and shallow analysis. We introduce branch-and-rethink (BR-RM), a two-turn RM that transfers the think-twice principle to reward modeling. Turn 1 performs adaptive branching, selecting a small set of instance-critical dimensions (such as factuality and safety) and sketching concise, evidence-seeking hypotheses. Turn 2 executes branch-conditioned rethinking, a targeted reread that tests those hypotheses and scrutinizes only what matters most. We train with GRPO-style reinforcement learning over structured two-turn traces using a simple binary outcome reward with strict format checks, making the approach compatible with standard RLHF pipelines. By converting all-at-once scoring into focused, second-look
reasoning, BR-RM reduces judgment
diffusion and improves sensitivity to subtle yet consequential errors while remaining practical and scalable. Experimental results demonstrate that our model achieves state-of-the-art performance on three challenging reward modeling
benchmarks across diverse domains.
Read the full abstract ▾
All authors Yizhu Jiao, Jiaqi Zeng, Julien Veron Vialard, Oleksii Kuchaiev, Jiawei Han, Olivier Delalleau
Computation and Language
Hope Speech Detection in Social Media English Corpora: Performance of Traditional and Transformer Models
Luis Ramos, Hiram Calvo and Olga Kolesnikova
The identification of hope
speech has become a promised NLP task, considering the need to detect motivational expressions of agency and goal-directed behaviour on social media platforms. This proposal evaluates traditional machine learning models and fine-tuned
transformers for a previously split hope
speech dataset as train, development and test set. On development test, a linear-kernel SVM and logistic regression both reached a macro-F1 of 0.78; SVM with RBF kernel reached 0.77, and Naïve Bayes hit 0.75.
Transformer models delivered better results, the best model achieved weighted precision of 0.82, weighted recall of 0.80, weighted F1 of 0.79, macro F1 of 0.79, and 0.80 accuracy. These results suggest that while optimally configured traditional machine learning models remain agile,
transformer architectures detect some subtle semantics of hope to achieve higher precision and recall in hope
speech detection, suggesting that larges
transformers and LLMs could perform better in small
datasets.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
ReCode: Unify Plan and Action for Universal Granularity Control
Zhaoyang Yu, Jiayi Zhang, Huixue Su et al. (+9 more)
Real-world tasks require decisions at varying granularities, and humans excel at this by leveraging a unified cognitive representation where planning is fundamentally understood as a high-level form of action. However, current Large
Language Model (LLM)-based
agents lack this crucial capability to operate fluidly across decision granularities. This limitation stems from existing paradigms that enforce a rigid separation between high-level planning and low-level action, which impairs dynamic adaptability and limits generalization. We propose ReCode (Recursive Code Generation), a novel paradigm that addresses this limitation by unifying planning and action within a single code representation. In this representation, ReCode treats high-level plans as abstract placeholder functions, which the
agent then recursively decomposes into finer-grained sub-functions until reaching primitive actions. This recursive approach dissolves the rigid boundary between plan and action, enabling the
agent to dynamically control its decision granularity. Furthermore, the recursive structure inherently generates rich, multi-granularity training data, enabling models to learn hierarchical decision-making processes. Extensive experiments show ReCode significantly surpasses advanced baselines in inference performance and demonstrates exceptional data efficiency in training, validating our core insight that unifying planning and action through recursive code generation is a powerful and effective approach to achieving universal granularity control. The code is available at https://github.com/FoundationAgents/ReCode.
Read the full abstract ▾
All authors Zhaoyang Yu, Jiayi Zhang, Huixue Su, Yufan Zhao, Yifan Wu, Mingyi Deng, Jinyu Xiang, Yizhang Lin, Lingxiao Tang, Yuyu Luo, Bang Liu, Chenglin Wu
Computation and Language · Cross-listed from cs.SD
ISA-Bench: Benchmarking Instruction Sensitivity for Large Audio Language Models
Bohan Li, Wenbin Huang, Yuhang Qiu et al. (+7 more)
Large Audio
Language Models (LALMs), which couple acoustic perception with large
language models (LLMs) to extract and understand diverse information from audio, have attracted intense interest from both academic and industrial communities. However, existing LALMs are highly sensitive to how instructions are phrased, affecting both (i) instruction-following rates and (ii) task performance. Yet, no existing
benchmarks offer a systematic and comprehensive evaluation of this sensitivity. We introduce ISA-Bench, a dynamic
benchmark evaluating instruction sensitivity for LALMs along three axes: instruction description, output format, and task composition. We assess recent open-source and proprietary LALMs using ISA-Bench, profiling both compliance and accuracy under controlled instruction variations. Experimental results reveal that even state-of-the-art LALMs suffer significant instruction sensitivity, leading to degraded performance on fundamental audio understanding tasks. To mitigate this issue, we fine-tune Qwen2-Audio on a specifically constructed complex instruction-variant
dataset, achieving a marked improvement in instruction-following performance. However, this also induces nontrivial catastrophic forgetting: the model loses some previously mastered task capabilities when exposed to new instruction styles. Our
benchmark provides a standardized basis for assessing and improving instruction sensitivity in LALMs, underscoring the need for instruction-robust audio understanding in real-world pipelines.
Read the full abstract ▾
All authors Bohan Li, Wenbin Huang, Yuhang Qiu, Yiwei Guo, Hankun Wang, Zhihan Li, Jing Peng, Ziyang Ma, Xie Chen, Kai Yu
Computation and Language · Cross-listed from cs.LG
Siddharth Sahay and Radhika Agarwal
This paper presents an end-to-end
multilingual translation pipeline that integrates a custom U-Net for text detection, the Tesseract engine for text recognition, and a from-scratch sequence-to-sequence (Seq2Seq)
Transformer for Neural Machine Translation (NMT). Our approach first utilizes a U-Net model, trained on a synthetic
dataset , to accurately segment and detect text regions from an image. These detected regions are then processed by Tesseract to extract the source text. This extracted text is fed into a custom
Transformer model trained from scratch on a
multilingual parallel corpus spanning 5
languages. Unlike systems reliant on monolithic pre-trained models, our architecture emphasizes full customization and adaptability. The system is evaluated on its text detection accuracy, text recognition quality, and translation performance via BLEU scores. The complete pipeline demonstrates promising results, validating the viability of a custom-built system for translating text directly from images.
Read the full abstract ▾
Computation and Language
LimRank: Less is More for Reasoning-Intensive Information Reranking
Tingyu Song, Yilun Zhao, Siyue Zhang et al. (+2 more)
Existing approaches typically rely on large-scale fine-tuning to adapt LLMs for information reranking tasks, which is computationally expensive. In this work, we demonstrate that modern LLMs can be effectively adapted using only minimal, high-quality supervision. To enable this, we design LIMRANK-SYNTHESIZER, a reusable and open-source pipeline for generating diverse, challenging, and realistic reranking examples. Using this synthetic data, we fine-tune our reranker model, LIMRANK. We evaluate LIMRANK on two challenging
benchmarks, i.e., BRIGHT for
reasoning-intensive
retrieval and FollowIR for instruction-following
retrieval. Our experiments demonstrate that LIMRANK achieves competitive performance, while being trained on less than 5% of the data typically used in prior work. Further ablation studies demonstrate the effectiveness of LIMRANK-SYNTHESIZER and the strong generalization capabilities of LIMRANK across downstream tasks, including scientific literature search and
retrieval-augmented generation for knowledge-intensive problem solving.
Read the full abstract ▾
All authors Tingyu Song, Yilun Zhao, Siyue Zhang, Chen Zhao, Arman Cohan
Computation and Language · Cross-listed from cs.AI
JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence
Qiushi Sun, Jingyang Gong, Yang Liu et al. (+6 more)
The scope of neural code intelligence is rapidly expanding beyond text-based source code to encompass the rich visual outputs that programs generate. This visual dimension is critical for advanced applications like flexible content generation and precise, program-driven editing of visualizations. However, progress has been impeded by the scarcity of high-quality
multimodal code data, a bottleneck stemming from challenges in synthesis and quality assessment. To address these challenges, we make contributions from both a data and modeling perspective. We first introduce a complete synthesis toolkit that leverages reciprocal synergies between data modalities to efficiently produce a large-scale, high-quality corpus spanning from standard charts to complex interactive web UIs and code-driven animations. Leveraging this toolkit, we construct JanusCode-800K, the largest
multimodal code corpus to date. This powers the training of our models, JanusCoder and JanusCoderV, which establish a visual-programmatic interface for generating code from textual instructions, visual inputs, or a combination of both. Our unified model is a departure from existing approaches that build specialized models for isolated tasks. Extensive experiments on both text-centric and
vision-centric coding tasks demonstrate the superior performance of the JanusCoder series, with our 7B to 14B scale models approaching or even exceeding the performance of commercial models. Furthermore, extensive analysis provides key insights into harmonizing programmatic logic with its visual expression. Our code and checkpoints are available at https://github.com/InternLM/JanusCoder.
Read the full abstract ▾
All authors Qiushi Sun, Jingyang Gong, Yang Liu, Qiaosheng Chen, Lei Li, Kai Chen, Qipeng Guo, Ben Kao, Fei Yuan
Computation and Language
IPQA: A Benchmark for Core Intent Identification in Personalized Question Answering
Jieyong Kim, Maryam Amirizaniani, Soojin Yoon et al. (+1 more)
Intent identification serves as the foundation for generating appropriate responses in personalized question answering (PQA). However, existing
benchmarks evaluate only response quality or
retrieval performance without directly measuring intent identification capabilities. This gap is critical because without understanding which intents users prioritize, systems cannot generate responses satisfying individual information needs. To address this, we introduce the concept of core intents: intents users prioritize when selecting answers to satisfy their information needs. To evaluate these core intents, we propose IPQA, a
benchmark for core Intent identification in Personalized Question Answering. Since users do not explicitly state their prioritized intents, we derive core intents from observable behavior patterns in answer selection, grounded in satisficing theory where users choose answers meeting their acceptance thresholds. We construct a
dataset with various domains through systematic filtering, LLM-based annotation, and rigorous quality control combining automated verification with human validation. Experimental evaluations across state-of-the-art
language models reveal that current systems struggle with core intent identification in personalized contexts. Models fail to identify core intents from user histories, with performance degrading as question complexity increases. The code and
dataset will be made publicly available to facilitate future research in this direction.
Read the full abstract ▾
All authors Jieyong Kim, Maryam Amirizaniani, Soojin Yoon, Dongha Lee
Computation and Language
M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset
Jiahui Geng, Jonathan Tonglet and Iryna Gurevych
Existing real-world
datasets for
multimodal fact-checking have multiple limitations: they contain few instances, cover only one or two
languages, focus on a single task, or rely on external news article sets to source true claims. To address these shortcomings, we introduce M4FC, a new real-world
dataset comprising 4,982 images paired with 6,980 claims. The images, verified by professional fact-checkers from 22 organizations, represent a diverse range of cultural and geographic contexts. Each claim is available in one or two out of ten
languages. M4FC spans six
multimodal fact-checking tasks: visual claim extraction, claimant intent prediction, fake image detection, image contextualization, location verification, and verdict prediction. We provide baseline results for all tasks and analyze how combining intermediate tasks affects verdict prediction performance. We make our
dataset and code publicly available.
Read the full abstract ▾
Computation and Language
MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring
Tengchao Yang, Sichen Guo, Mengzhao Jia et al. (+4 more)
Effective math tutoring requires not only solving problems but also diagnosing students' difficulties and guiding them step by step. While
multimodal large
language models (MLLMs) show promise, existing
benchmarks largely overlook these tutoring skills. We introduce MMTutorBench, the first
benchmark for AI math tutoring, consisting of 685 problems built around pedagogically significant key-steps. Each problem is paired with problem-specific rubrics that enable fine-grained evaluation across six dimensions, and structured into three tasks-Insight Discovery, Operation Formulation, and Operation Execution. We evaluate 12 leading MLLMs and find clear performance gaps between proprietary and open-source systems, substantial room compared to human tutors, and consistent trends across input variants: OCR pipelines degrade tutoring quality, few-shot prompting yields limited gains, and our rubric-based LLM-as-a-Judge proves highly reliable. These results highlight both the difficulty and diagnostic value of MMTutorBench for advancing AI tutoring.
Read the full abstract ▾
All authors Tengchao Yang, Sichen Guo, Mengzhao Jia, Jiaming Su, Yuanyang Liu, Zhihan Zhang, Meng Jiang
Computation and Language
Evaluating Large Language Models for Stance Detection on Financial Targets from SEC Filing Reports and Earnings Call Transcripts
Nikesh Gyawali, Doina Caragea, Alex Vasenkov et al. (+1 more)
Financial narratives from U.S. Securities and Exchange Commission (SEC) filing reports and quarterly earnings call transcripts (ECTs) are very important for investors, auditors, and regulators. However, their length, financial jargon, and nuanced
language make fine-grained analysis difficult. Prior sentiment analysis in the financial domain required a large, expensive labeled
dataset, making the sentence-level stance towards specific financial targets challenging. In this work, we introduce a sentence-level corpus for stance detection focused on three core financial metrics: debt, earnings per share (EPS), and sales. The sentences were extracted from Form 10-K annual reports and ECTs, and labeled for stance (positive, negative, neutral) using the advanced ChatGPT-o3-pro model under rigorous human validation. Using this corpus, we conduct a systematic evaluation of modern large
language models (LLMs) using zero-shot, few-shot, and Chain-of-Thought (CoT) prompting strategies. Our results show that few-shot with CoT prompting performs best compared to supervised baselines, and LLMs' performance varies across the SEC and ECT
datasets. Our findings highlight the practical viability of leveraging LLMs for target-specific stance in the financial domain without requiring extensive labeled data.
Read the full abstract ▾
All authors Nikesh Gyawali, Doina Caragea, Alex Vasenkov, Cornelia Caragea
Computation and Language
BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents
Litu Ou, Kuan Li, Huifeng Yin et al. (+8 more)
Confidence in LLMs is a useful indicator of model uncertainty and answer reliability. Existing work mainly focused on single-turn scenarios, while research on confidence in complex multi-turn interactions is limited. In this paper, we investigate whether LLM-based search
agents have the ability to communicate their own confidence through verbalized confidence scores after long sequences of actions, a significantly more challenging task compared to outputting confidence in a single interaction. Experimenting on open-source agentic models, we first find that models exhibit much higher task accuracy at high confidence while having near-zero accuracy when confidence is low. Based on this observation, we propose Test-Time Scaling (TTS) methods that use confidence scores to determine answer quality, encourage the model to try again until reaching a satisfactory confidence level. Results show that our proposed methods significantly reduce
token consumption while demonstrating competitive performance compared to baseline fixed budget TTS methods.
Read the full abstract ▾
All authors Litu Ou, Kuan Li, Huifeng Yin, Liwen Zhang, Zhongwang Zhang, Xixi Wu, Rui Ye, Zile Qiao, Pengjun Xie, Jingren Zhou, Yong Jiang
Computation and Language
Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences
Zhuoran Jin, Hongbang Yuan, Kejian Zhu et al. (+5 more)
Reward models (RMs) play a critical role in aligning AI behaviors with human preferences, yet they face two fundamental challenges: (1) Modality Imbalance, where most RMs are mainly focused on text and image modalities, offering limited support for
video, audio, and other modalities; and (2) Preference
Rigidity, where training on fixed binary preference pairs fails to capture the complexity and diversity of personalized preferences. To address the above challenges, we propose Omni-Reward, a step toward generalist omni-modal reward modeling with support for free-form preferences, consisting of: (1) Evaluation: We introduce Omni-RewardBench, the first omni-modal RM
benchmark with free-form preferences, covering nine tasks across five modalities including text, image,
video, audio, and 3D; (2) Data: We construct Omni-RewardData, a
multimodal preference
dataset comprising 248K general preference pairs and 69K instruction-tuning pairs for training generalist omni-modal RMs; (3) Model: We propose Omni-RewardModel, which includes both discriminative and generative RMs, and achieves strong performance on Omni-RewardBench as well as other widely used reward modeling
benchmarks.
Read the full abstract ▾
All authors Zhuoran Jin, Hongbang Yuan, Kejian Zhu, Jiachun Li, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao
Computation and Language · Cross-listed from cs.AI
A Neuro-Symbolic Multi-Agent Approach to Legal-Cybersecurity Knowledge Integration
Chiara Bonfanti, Alessandro Druetto, Cataldo Basile et al. (+2 more)
The growing intersection of cybersecurity and law creates a complex information space where traditional legal research tools struggle to deal with nuanced connections between cases, statutes, and technical vulnerabilities. This knowledge divide hinders collaboration between legal experts and cybersecurity professionals. To address this important gap, this work provides a first step towards intelligent systems capable of navigating the increasingly intricate cyber-legal domain. We demonstrate promising initial results on
multilingual tasks.
Read the full abstract ▾
All authors Chiara Bonfanti, Alessandro Druetto, Cataldo Basile, Tharindu Ranasinghe, Marcos Zampieri
Computation and Language
EMTSF:Extraordinary Mixture of SOTA Models for Time Series Forecasting
Musleh Alharthi, Kaleel Mahmood, Sarosh Patel et al. (+1 more)
The immense success of the
Transformer architecture in Natural
Language Processing has led to its adoption in Time Se ries
Forecasting (TSF), where superior performance has been shown. However, a recent important paper questioned their effectiveness by demonstrating that a simple single layer linear model outperforms
Transformer-based models. This was soon shown to be not as valid, by a better
transformer-based model termed PatchTST. More re cently, TimeLLM demonstrated even better results by repurposing a Large
Language Model (LLM) for the TSF domain. Again, a follow up paper challenged this by demonstrating that removing the LLM component or replacing it with a basic
attention layer in fact yields better performance. One of the challenges in
forecasting is the fact that TSF data favors the more recent past, and is sometimes subject to unpredictable events. Based upon these recent insights in TSF, we propose a strong Mixture of Experts (MoE) framework. Our method combines the state-of-the-art (SOTA) models including xLSTM, en hanced Linear, PatchTST, and minGRU, among others. This set of complimentary and diverse models for TSF are integrated in a Trans former based MoE gating network. Our proposed model outperforms all existing TSF models on standard
benchmarks, surpassing even the latest approaches based on MoE frameworks.
Read the full abstract ▾
All authors Musleh Alharthi, Kaleel Mahmood, Sarosh Patel, Ausif Mahmood
Computation and Language
Detecting Religious Language in Climate Discourse
Evy Beijen, Pien Pieterse, Yusuf Çelik et al. (+3 more)
Religious
language continues to permeate contemporary discourse, even in ostensibly secular domains such as environmental activism and
climate change debates. This paper investigates how explicit and implicit forms of religious
language appear in
climate-related texts produced by secular and religious nongovernmental organizations (NGOs). We introduce a dual methodological approach: a rule-based model using a hierarchical tree of religious terms derived from ecotheology literature, and large
language models (LLMs) operating in a zero-shot setting. Using a
dataset of more than 880,000 sentences, we compare how these methods detect religious
language and analyze points of agreement and divergence. The results show that the rule-based method consistently labels more sentences as religious than LLMs. These findings highlight not only the methodological challenges of computationally detecting religious
language but also the broader tension over whether religious
language should be defined by vocabulary alone or by contextual meaning. This study contributes to digital methods in religious studies by demonstrating both the potential and the limitations of approaches for analyzing how the sacred persists in
climate discourse.
Read the full abstract ▾
All authors Evy Beijen, Pien Pieterse, Yusuf Çelik, Willem Th. van Peursen, Sandjai Bhulai, Meike Morren
Computation and Language
How AI Forecasts AI Jobs: Benchmarking LLM Predictions of Labor Market Changes
Sheri Osborn, Rohit Valecha, H. Raghav Rao et al. (+2 more)
Artificial intelligence is reshaping labor
markets, yet we lack tools to systematically forecast its effects on employment. This paper introduces a
benchmark for evaluating how well large
language models (LLMs) can anticipate changes in job demand, especially in occupations affected by AI. Existing research has shown that LLMs can extract sentiment, summarize economic reports, and emulate forecaster behavior, but little work has assessed their use for forward-looking labor prediction. Our
benchmark combines two complementary
datasets: a high-frequency index of sector-level job postings in the United States, and a global
dataset of projected occupational changes due to AI adoption. We format these data into
forecasting tasks with clear temporal splits, minimizing the risk of information leakage. We then evaluate LLMs using multiple prompting strategies, comparing task-scaffolded, persona-driven, and hybrid approaches across model families. We assess both quantitative accuracy and qualitative consistency over time. Results show that structured task
prompts consistently improve forecast stability, while persona
prompts offer advantages on short-term trends. However, performance varies significantly across sectors and horizons, highlighting the need for domain-aware prompting and rigorous evaluation protocols. By releasing our
benchmark, we aim to support future research on labor
forecasting,
prompt design, and LLM-based economic
reasoning. This work contributes to a growing body of research on how LLMs interact with real-world economic data, and provides a reproducible testbed for studying the limits and opportunities of AI as a
forecasting tool in the context of labor
markets.
Read the full abstract ▾
All authors Sheri Osborn, Rohit Valecha, H. Raghav Rao, Dan Sass, Anthony Rios
Computation and Language
LightKGG: Simple and Efficient Knowledge Graph Generation from Textual Data
Teng Lin
The scarcity of high-quality knowledge graphs (KGs) remains a critical bottleneck for downstream AI applications, as existing extraction methods rely heavily on error-prone pattern-matching techniques or resource-intensive large
language models (LLMs). While recent tools leverage LLMs to generate KGs, their computational demands limit accessibility for low-resource environments. Our paper introduces LightKGG, a novel framework that enables efficient KG extraction from textual data using small-scale
language models (SLMs) through two key technical innovations: (1) Context-integrated Graph extraction integrates contextual information with nodes and edges into a unified graph structure, reducing the reliance on complex semantic processing while maintaining more key information; (2)
Topology-enhanced relationship inference leverages the inherent
topology of the extracted graph to efficiently infer relationships, enabling relationship discovery without relying on complex
language understanding capabilities of LLMs. By enabling accurate KG construction with minimal hardware requirements, this work bridges the gap between automated knowledge extraction and practical deployment scenarios while introducing scientifically rigorous methods for optimizing SLM efficiency in structured NLP tasks.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Planning Ahead with RSA: Efficient Signalling in Dynamic Environments by Projecting User Awareness across Future Timesteps
Anwesha Das, John Duff, Jörg Hoffmann et al. (+1 more)
Adaptive
agent design offers a way to improve human-AI collaboration on time-sensitive tasks in rapidly changing environments. In such cases, to ensure the human maintains an accurate understanding of critical task elements, an assistive
agent must not only identify the highest priority information but also estimate how and when this information can be communicated most effectively, given that human
attention represents a zero-sum cognitive resource where focus on one message diminishes awareness of other or upcoming information. We introduce a theoretical framework for adaptive signalling which meets these challenges by using principles of rational communication, formalised as
Bayesian reference resolution using the Rational
Speech Act (RSA) modelling framework, to plan a sequence of messages which optimise timely
alignment between user belief and a dynamic environment. The
agent adapts message specificity and timing to the particulars of a user and scenario based on projections of how prior-guided interpretation of messages will influence
attention to the interface and subsequent belief update, across several timesteps out to a fixed horizon. In a comparison to baseline methods, we show that this effectiveness depends crucially on combining multi-step planning with a realistic model of user awareness. As the first application of RSA for communication in a dynamic environment, and for human-AI interaction in general, we establish theoretical foundations for pragmatic communication in human-
agent teams, highlighting how insights from cognitive science can be capitalised to inform the design of assistive
agents.
Read the full abstract ▾
All authors Anwesha Das, John Duff, Jörg Hoffmann, Vera Demberg
Computation and Language
BaZi-Based Character Simulation Benchmark: Evaluating AI on Temporal and Persona Reasoning
Siyuan Zheng, Pai Liu, Xi Chen et al. (+2 more)
Human-like virtual characters are crucial for games, storytelling, and virtual reality, yet current methods rely heavily on annotated data or handcrafted persona
prompts, making it difficult to scale up and generate realistic, contextually coherent personas. We create the first QA
dataset for BaZi-based persona
reasoning, where real human experiences categorized into wealth, health, kinship, career, and relationships are represented as life-event questions and answers. Furthermore, we propose the first BaZi-LLM system that integrates symbolic
reasoning with large
language models to generate temporally dynamic and fine-grained virtual personas. Compared with mainstream LLMs such as DeepSeek-v3 and GPT-5-mini, our method achieves a 30.3%-62.6% accuracy improvement. In addition, when incorrect BaZi information is used, our model's accuracy drops by 20%-45%, showing the potential of culturally grounded symbolic-LLM integration for realistic character simulation.
Read the full abstract ▾
All authors Siyuan Zheng, Pai Liu, Xi Chen, Jizheng Dong, Sihan Jia
Computation and Language
Adaptive Blockwise Search: Inference-Time Alignment for Large Language Models
Mohammad Atif Quamar, Mohammad Areeb, Nishant Sharma et al. (+5 more)
LLM
alignment remains a critical challenge. Inference-time methods provide a flexible alternative to fine-tuning, but their uniform computational effort often yields suboptimal
alignment. We hypothesize that for many
alignment tasks, the initial
tokens of a response are disproportionately more critical. To leverage this principle, we introduce AdaSearch, a novel blockwise search strategy. It adaptively allocates a fixed computational budget using a sampling schedule, focusing search effort on these critical
tokens. We apply AdaSearch to sequential decoding and introduce its tree-search counterpart, AdaBeam. Our comprehensive evaluation across eight LLMs demonstrates that AdaSearch outperforms strong Best-of-N and fine-tuning baselines. Specifically, win-rates improve by over 10% for harmlessness generation, controlled sentiment generation, and for mathematical
reasoning tasks relative to Best-of-N.
Read the full abstract ▾
All authors Mohammad Atif Quamar, Mohammad Areeb, Nishant Sharma, Ananth Shreekumar, Jonathan Rosenthal, Muslum Ozgur Ozmen, Mikhail Kuznetsov, Z. Berkay Celik
Computation and Language · Cross-listed from eess.AS
LibriConvo: Simulating Conversations from Read Literature for ASR and Diarization
Máté Gedeon and Péter Mihajlik
We introduce LibriConvo, a synthetic conversational
speech corpus for speaker diarization and automatic
speech recognition (ASR), built by instantiating the previously proposed Speaker-Aware Simulated Conversation (SASC) framework in a
dataset and benchmarking setting. The main contribution of this paper is a corpus construction pipeline and
benchmark derived from that framework. To make the data more suitable for downstream ASR and diarization, conversational timing statistics are estimated from English CallHome using external voice activity detection, long pauses are compressed, LibriTTS utterances are grouped by book to improve local semantic continuity, and room impulse responses are selected with a spatial-plausibility heuristic. The resulting corpus contains 240.1 hours of audio across 1,496 dialogues involving 830 speakers, partitioned into speaker-disjoint train, validation, and test splits. We report baseline results for both diarization and ASR. On the test split, Sortformer outperforms the pyannote pipeline in diarization (11.1% vs.~24.4% DER). For ASR, a Fast Conformer-CTC XLarge model fine-tuned with Serialized Output Training achieves 7.29% WER and 6.97% cpWER, outperforming zero-shot Whisper-large-v3. These results position LibriConvo as a practical
benchmark for studying synthetic conversational
speech and for evaluating multi-speaker
speech processing systems.
Read the full abstract ▾
Computation and Language
Arabic Little STT: Arabic Children Speech Recognition Dataset
Mouhand Alkadri, Dania Desouki and Khloud Al Jallad
The performance of Artificial Intelligence (AI) systems fundamentally depends on high-quality training data. However, low-resource
languages like Arabic suffer from severe data scarcity. Moreover, the absence of child-specific
speech corpora is an essential gap that poses significant challenges. To address this gap, we present our created
dataset, Arabic Little STT, a
dataset of Levantine Arabic child
speech recorded in classrooms, containing 355 utterances from 288 children (ages 6 - 13). We further conduct a systematic assessment of Whisper, a state-of-the-art automatic
speech recognition (ASR) model, on this
dataset and compare its performance with adult Arabic
benchmarks. Our evaluation across eight Whisper variants reveals that even the best-performing model (Large_v3) struggles significantly, achieving a 0.66 word error rate (WER) on child
speech, starkly contrasting with its sub 0.20 WER on adult
datasets. These results align with other research on English
speech. Results highlight the critical need for dedicated child
speech benchmarks and inclusive training data in ASR development. Emphasizing that such data must be governed by strict ethical and
privacy frameworks to protect sensitive child information. We hope that this study provides an initial step for future work on equitable
speech technologies for Arabic-speaking children. We hope that our publicly available
dataset enrich the children's demographic representation in ASR
datasets.
Read the full abstract ▾