Computation and Language
Tackling a Challenging Corpus for Early Detection of Gambling Disorder: UNSL at MentalRiskES 2025
Horacio Thompson and Marcelo Errecalde
Gambling disorder is a complex behavioral addiction that is challenging to understand and address, with severe physical, psychological, and social consequences. Early Risk Detection (ERD) on the Web has become a key task in the scientific community for identifying early signs of mental health behaviors based on social media activity. This work presents our participation in the MentalRiskES 2025 challenge, specifically in Task 1, aimed at classifying users at high or low risk of developing a gambling-related disorder. We proposed three methods based on a CPI+DMC approach, addressing predictive effectiveness and decision-making speed as independent objectives. The components were implemented using the SS3, BERT with extended vocabulary, and SBERT models, followed by decision policies based on historical user analysis. Although it was a challenging corpus, two of our proposals achieved the top two positions in the official results, performing notably in decision metrics. Further analysis revealed some difficulty in distinguishing between users at high and low risk, reinforcing the need to explore strategies to improve data interpretation and quality, and to promote more transparent and reliable ERD systems for mental disorders.
Read the full abstract ▾
Computation and Language
Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models
Xiang Hu, Zhanchao Zhou, Ruiqi Liang et al. (+3 more)
This work explores the challenge of building “Machines that Can Remember”, framing long-term
memory as the problem of efficient ultra-long context modeling. We argue that this requires three key properties: sparsity, random-access flexibility, and length generalization. To address ultra-long-context modeling, we leverage Hierarchical Sparse
Attention (HSA), a novel
attention mechanism that satisfies all three properties. We integrate HSA into
Transformers to build HSA-UltraLong, which is an 8B-parameter MoE model trained on over 8 trillion
tokens and is rigorously evaluated on different tasks with in-domain and out-of-domain context lengths to demonstrate its capability in handling ultra-long contexts. Results show that our model performs comparably to full-
attention baselines on in-domain lengths while achieving over 90% accuracy on most in-context
retrieval tasks with contexts up to 16M. This report outlines our experimental insights and open problems, contributing a foundation for future research in ultra-long context modeling.
Read the full abstract ▾
All authors Xiang Hu, Zhanchao Zhou, Ruiqi Liang, Zehuan Li, Wei Wu, Jianguo Li
Computation and Language · Cross-listed from cs.CV
Toward Automatic Safe Driving Instruction: A Large-Scale Vision Language Model Approach
Haruki Sakajo, Hiroshi Takato, Hiroshi Tsutsui et al. (+3 more)
Large-scale
Vision Language Models (LVLMs) exhibit advanced capabilities in tasks that require visual information, including object detection. These capabilities have promising applications in various industrial domains, such as autonomous driving. For example, LVLMs can generate safety-oriented descriptions of
videos captured by road-facing cameras. However, ensuring comprehensive safety requires monitoring driver-facing views as well to detect risky events, such as the use of mobiles while driving. Thus, the ability to process synchronized inputs is necessary from both driver-facing and road-facing cameras. In this study, we develop models and investigate the capabilities of LVLMs by constructing a
dataset and evaluating their performance on this
dataset. Our experimental results demonstrate that while pre-trained LVLMs have limited effectiveness, fine-tuned LVLMs can generate accurate and safety-aware driving instructions. Nonetheless, several challenges remain, particularly in detecting subtle or complex events in the
video. Our findings and error analysis provide valuable insights that can contribute to the improvement of LVLM-based systems in this domain.
Read the full abstract ▾
All authors Haruki Sakajo, Hiroshi Takato, Hiroshi Tsutsui, Komei Soda, Hidetaka Kamigaito, Taro Watanabe
Computation and Language · Cross-listed from cs.LG
Transformer-Driven Triple Fusion Framework for Enhanced Multimodal Author Intent Classification in Low-Resource Bangla
Ariful Islam, Tanvir Mahmud and Md Rifat Hossen
The expansion of the Internet and social networks has led to an explosion of user-generated content. Author intent understanding plays a crucial role in interpreting social media content. This paper addresses author intent classification in Bangla social media posts by leveraging both textual and visual data. Recognizing limitations in previous unimodal approaches, we systematically
benchmark transformer-based
language models (mBERT, DistilBERT, XLM-RoBERTa) and
vision architectures (ViT, Swin, SwiftFormer, ResNet, DenseNet, MobileNet), utilizing the Uddessho
dataset of 3,048 posts spanning six practical intent categories. We introduce a novel intermediate fusion strategy that significantly outperforms early and late fusion on this task. Experimental results show that intermediate fusion, particularly with mBERT and Swin
Transformer, achieves 84.11% macro-F1 score, establishing a new state-of-the-art with an 8.4 percentage-point improvement over prior Bangla
multimodal approaches. Our analysis demonstrates that integrating visual context substantially enhances intent classification. Cross-modal feature integration at intermediate levels provides optimal balance between modality-specific representation and cross-modal learning. This research establishes new
benchmarks and methodological standards for Bangla and other low-resource
languages. We call our proposed framework BangACMM (Bangla Author Content
MultiModal).
Read the full abstract ▾
Computation and Language
MCP vs RAG vs NLWeb vs HTML: A Comparison of the Effectiveness and Efficiency of Different Agent Interfaces to the Web (Technical Report)
Aaron Steiner, Ralph Peeters and Christian Bizer
Large
language model
agents are increasingly used to automate web tasks such as product search, offer comparison, and checkout. Current research explores different interfaces through which these
agents interact with websites, including traditional HTML browsing,
retrieval-augmented generation (RAG) over pre-crawled content, communication via Web APIs using the Model Context Protocol (MCP), and natural-
language querying through the NLWeb interface. However, no prior work has compared these four architectures within a single controlled environment using identical tasks. To address this gap, we introduce a testbed consisting of four simulated e-shops, each offering its products via HTML, MCP, and NLWeb interfaces. For each interface (HTML, RAG, MCP, and NLWeb) we develop specialized
agents that perform the same sets of tasks, ranging from simple product searches and price comparisons to complex queries for complementary or substitute products and checkout processes. We evaluate the
agents using GPT 4.1, GPT 5, GPT 5 mini, and Claude Sonnet 4 as underlying LLM. Our evaluation shows that the RAG, MCP and NLWeb
agents outperform HTML on both effectiveness and efficiency. Averaged over all tasks, F1 rises from 0.67 for HTML to between 0.75 and 0.77 for the other
agents.
Token usage falls from about 241k for HTML to between 47k and 140k per task. The runtime per task drops from 291 seconds to between 50 and 62 seconds. The best overall configuration is RAG with GPT 5 achieving an F1 score of 0.87 and a completion rate of 0.79. Also taking cost into consideration, RAG with GPT 5 mini offers a good compromise between API usage fees and performance. Our experiments show the choice of the interaction interface has a substantial impact on both the effectiveness and efficiency of LLM-based web
agents.
Read the full abstract ▾
Computation and Language
Learning a Single Token to Replace Long System Prompts in LLMs
Jiancheng Dong, Pengyue Jia, Jingyu Peng et al. (+7 more)
Long system
prompts are widely used to steer Large
Language Models (LLMs), but repeatedly processing them at inference time is inefficient and consumes valuable context budget. This motivates a central question: can the behavioral effect of a long system
prompt be retained using only a minimal learned representation? To enable this, we propose a lightweight training framework that learns a single Behavior-Equivalent
Token ([BE]). The framework first trains [BE] to encode the semantic content of the original system
prompt via reconstruction, and then distills the
prompt's downstream behavior into this single
token. Importantly, our method requires no update to the pretrained LLM weights, no auxiliary compression models, and no labeled responses. Empirical evaluations on three
datasets show that replacing long
prompts with a single [BE]
token yields up to a $3000\times$
prompt compression ratio, while retaining about 98% of the downstream performance of the original system
prompts. This substantially reduces inference cost and frees nearly the entire context window for user inputs and model outputs.
Read the full abstract ▾
All authors Jiancheng Dong, Pengyue Jia, Jingyu Peng, Maolin Wang, Yuhao Wang, Lixin Su, Xin Sun, Shuaiqiang Wang, Dawei Yin, Xiangyu Zhao
Computation and Language · Cross-listed from cs.LG
BanglaSentNet: An Explainable Hybrid Deep Learning Framework for Multi-Aspect Sentiment Analysis with Cross-Domain Transfer Learning
Ariful Islam, Md Rifat Hossen and Tanvir Mahmud
Multi-aspect sentiment analysis of Bangla e-commerce reviews remains challenging due to limited annotated
datasets, morphological complexity, code-mixing phenomena, and domain shift issues, affecting 300 million Bangla-speaking users. Existing approaches lack explainability and cross-domain generalization capabilities crucial for practical deployment. We present BanglaSentNet, an explainable hybrid deep learning framework integrating LSTM, BiLSTM, GRU, and BanglaBERT through dynamic weighted ensemble learning for multi-aspect sentiment classification. We introduce a
dataset of 8,755 manually annotated Bangla product reviews across four aspects (Quality, Service, Price, Decoration) from major Bangladeshi e-commerce platforms. Our framework incorporates SHAP-based feature attribution and
attention visualization for transparent insights. BanglaSentNet achieves 85% accuracy and 0.88 F1-score, outperforming standalone deep learning models by 3-7% and traditional approaches substantially. The explainability suite achieves 9.4/10 interpretability score with 87.6% human agreement. Cross-domain transfer learning experiments reveal robust generalization: zero-shot performance retains 67-76% effectiveness across diverse domains (BanglaBook reviews, social media, general e-commerce, news headlines); few-shot learning with 500-1000 samples achieves 90-95% of full fine-tuning performance, significantly reducing annotation costs. Real-world deployment demonstrates practical utility for Bangladeshi e-commerce platforms, enabling data-driven decision-making for pricing optimization, service improvement, and customer experience enhancement. This research establishes a new state-of-the-art
benchmark for Bangla sentiment analysis, advances ensemble learning methodologies for low-resource
languages, and provides actionable solutions for commercial applications.
Read the full abstract ▾
Computation and Language
Tourism Question Answer System in Indian Language using Domain-Adapted Foundation Models
Praveen Gatla, Anushka, Nikita Kanwar et al. (+2 more)
This article presents the first comprehensive study on designing a baseline extractive question-answering (QA) system for the Hindi tourism domain, with a specialized focus on the Varanasi-a cultural and spiritual hub renowned for its Bhakti-Bhaav (devotional ethos). Targeting ten tourism-centric subdomains-Ganga Aarti, Cruise, Food Court, Public Toilet, Kund, Museum, General, Ashram, Temple and Travel, the work addresses the absence of
language-specific QA resources in Hindi for culturally nuanced applications. In this paper, a
dataset comprising 7,715 Hindi QA pairs pertaining to Varanasi tourism was constructed and subsequently augmented with 27,455 pairs generated via Llama zero-shot prompting. We propose a framework leveraging foundation models-BERT and RoBERTa, fine-tuned using Supervised Fine-Tuning (SFT) and Low-Rank Adaptation (LoRA), to optimize parameter efficiency and task performance. Multiple variants of BERT, including pre-trained
languages (e.g., Hindi-BERT), are evaluated to assess their suitability for low-resource domain-specific QA. Evaluation metrics - F1, BLEU, and ROUGE-L - highlight trade-offs between answer precision and linguistic fluency. Experiments demonstrate that LoRA-based fine-tuning achieves competitive performance (85.3% F1) while reducing trainable parameters by 98% compared to SFT, striking a balance between efficiency and accuracy. Comparative analysis across models reveals that RoBERTa with SFT outperforms BERT variants in capturing contextual nuances, particularly for culturally embedded terms (e.g., Aarti, Kund). This work establishes a foundational baseline for Hindi tourism QA systems, emphasizing the role of LORA in low-resource settings and underscoring the need for culturally contextualized NLP frameworks in the tourism domain.
Read the full abstract ▾
All authors Praveen Gatla, Anushka, Nikita Kanwar, Gouri Sahoo, Rajesh Kumar Mundotiya
Computation and Language
TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies
Guang Liang, Jie Shao, Ningyuan Tang et al. (+2 more)
Native FP8 support in modern hardware is essential for training large
Transformers, but is severely hindered by extreme activation outliers. Existing solutions either rely on complex mixed-precision engineering or invasive architectural modifications. This paper fundamentally challenges the conventional wisdom that outliers are data-driven. We demonstrate that extreme outliers are a data-independent, mechanically-produced artifact of training, originating from specific structural properties of the weight
matrices (i.e., colinearity). Based on this insight, we propose TWEO (
Transformers Without Extreme Outliers), a novel, non-invasive loss function. TWEO effectively prevents extreme outliers via a very simple loss term, which reduces outliers from 10000+ to less than 20. TWEO then enables full-model FP8 pre-training with neither engineering tricks nor architectural changes for both LLM and ViT. When standard FP8 training catastrophically collapses, TWEO achieves performance comparable to the BF16 baseline while delivering a 36% increase in training throughput. Also, TWEO enables a new quantization paradigm. Hardware-friendly W8A8 per-tensor static quantization of LLMs, previously considered completely unusable due to outliers, achieves SOTA performance for the first time on TWEO-trained models.
Read the full abstract ▾
All authors Guang Liang, Jie Shao, Ningyuan Tang, Xinyao Liu, Jianxin Wu
Computation and Language
Listwise Preference Optimization with Element-wise Confusions for Aspect Sentiment Quad Prediction
Wenna Lai, Haoran Xie, Guandong Xu et al. (+2 more)
Aspect sentiment quad prediction (ASQP) is inherently challenging to predict a structured quadruple with four core sentiment elements, including aspect term (a), aspect category (c), opinion term (o), and sentiment polarity (s). Prior methods relying on marker-based prediction struggle with modeling the intricate relationships among elements and experience sharp performance declines when predicting higher-order elements (e.g., c and s) under standard supervised fine-tuning. To address these limitations, we employ
reasoning-based generation to output both the quadruple and a natural
language rationale under element prefixes within a unified template, encouraging explicit relational
reasoning and interpretability. To further enhance element-wise
alignment, we introduce a listwise preference optimization framework for improving structural validity and relational coherence. Specifically, we generate element-wise confusable candidates via syntactic and semantic proximity, then train the model with listwise objectives to prefer the gold candidates over closely competing alternatives. Extensive experiments on four
benchmark datasets demonstrate that our framework effectively improves quadruple prediction accuracy and explanation consistency.
Read the full abstract ▾
All authors Wenna Lai, Haoran Xie, Guandong Xu, Qing Li, S. Joe Qin
Computation and Language
Are LLMs Good Safety Agents or a Propaganda Engine?
Neemesh Yadav, Francesco Ortu, Jiarui Liu et al. (+5 more)
Large
Language Models (LLMs) are trained to refuse to respond to harmful content. However, systematic analyses of whether this behavior is truly a reflection of its safety policies or an indication of political censorship, that is practiced globally by countries, is lacking. Differentiating between safety influenced refusals or politically motivated censorship is hard and unclear. For this purpose we introduce PSP, a
dataset built specifically to probe the refusal behaviors in LLMs from an explicitly political context. PSP is built by formatting existing censored content from two data sources, openly available on the internet: sensitive
prompts in China generalized to multiple countries, and tweets that have been censored in various countries. We study: 1) impact of political sensitivity in seven LLMs through data-driven (making PSP implicit) and representation-level approaches (erasing the concept of politics); and, 2) vulnerability of models on PSP through
prompt injection attacks (PIAs). Associating censorship with refusals on content with masked implicit intent, we find that most LLMs perform some form of censorship. We conclude with summarizing major attributes that can cause a shift in refusal distributions across models and contexts of different countries.
Read the full abstract ▾
All authors Neemesh Yadav, Francesco Ortu, Jiarui Liu, Joeun Yook, Bernhard Schölkopf, Rada Mihalcea, Alberto Cazzaniga, Zhijing Jin
Computation and Language
Multi-chain Graph Refinement and Selection for Reliable Reasoning in Large Language Models
Yujiao Yang, Jing Lian and Linhui Li
The complex
reasoning ability of Large
Language Models (LLMs) poses a critical bottleneck for their practical applications. Test-time expansion methods such as Tree-of-Thought (ToT) and Graph-of-Thought (GoT) enhance
reasoning by introducing intermediate
reasoning structures, tree search, or graph-based exploration mechanisms. However, their
reasoning strategies suffer from limited diversity, redundant search branches, and inadequate integration and error correction across heterogeneous
reasoning paths. To address these limitations, we propose a novel
reasoning framework called Multi-chain Graph Refinement & Selection (MGRS), which first generates multiple diverse
reasoning trajectories for a given problem, refines candidate responses using a composite self- and cross-verification strategy, then constructs a
reasoning relation graph and estimates the success rate of intermediate nodes, and finally computes cumulative success rates to select the most reliable answer and corresponding
reasoning trajectory. Experimental results demonstrate that MGRS significantly advances both the
reasoning capability and computational efficiency of
reasoning enhancement methods. Across six
benchmark datasets spanning four distinct tasks, MGRS achieves an average accuracy of 82.9%, outperforming state-of-the-art baselines by a clear margin of 2.1%. Remarkably, on the 24-point game, MGRS attains 100% accuracy for the first time, while delivering a 13.6x speed-up compared to the leading Forest of Thoughts framework.
Read the full abstract ▾
Computation and Language
Dripper: Token-Efficient Main HTML Extraction with a Lightweight LM
Mengjie Liu, Jiahui Peng, Wenchang Ning et al. (+14 more)
High-quality main content extraction from web pages is a critical prerequisite for constructing large-scale training corpora. While traditional heuristic extractors are efficient, they lack the semantic
reasoning required to handle the structural heterogeneity of the modern web. Conversely, well-pretrained generative Large
Language Models (LLMs) offer superior document comprehension but are prohibited by excessive computational costs, limited context windows, and
hallucination risks when applied at web scale. We present Dripper, a lightweight framework that resolves these bottlenecks through four contributions: (1) We reformulate extraction as a constrained sequence labeling task using SLMs (Small
Language Models). This paradigm eliminates generative
hallucinations and achieves exceptional efficiency, reaching a throughput of 3.08 pages per second on a single A100 GPU. (2) We construct WebMainBench, a rigorous
benchmark of 7,809 human-annotated pages covering 5,434 unique domains and multiple
languages. Evaluations show our Dripper-0.6B model outperforms heuristics like Trafilatura and rivals massive models like DeepSeek-V3.2(685B), GPT-5 and Gemini-2.5-Pro, offering an optimal efficiency-accuracy trade-off. (3) We demonstrate infrastructural value by pre-training a 1B model on a Dripper-curated corpus (63B
tokens). This model significantly outperforms baselines in downstream tasks, proving the critical role of extraction quality and the effectiveness of our framework. (4) We open-source the Dripper-0.6B weights and codebase to facilitate the construction of high-quality
datasets.
Read the full abstract ▾
All authors Mengjie Liu, Jiahui Peng, Wenchang Ning, Pei Chu, Jiantao Qiu, Ren Ma, He Zhu, Rui Min, Lindong Lu, Linfeng Hou, Kaiwen Liu, Yuan Qu, Zhenxiang Li, Chao Xu, Zhongying Tu, Wentao Zhang, Conghui He
Computation and Language
Mind Reading or Misreading? LLMs on the Big Five Personality Test
Francesco Di Cursi, Chiara Boldrini, Marco Conti et al. (+1 more)
We evaluate large
language models (LLMs) for automatic personality prediction from text under the binary Five Factor Model (BIG5). Five models – including GPT-4 and lightweight open-source alternatives – are tested across three heterogeneous
datasets (Essays, MyPersonality, Pandora) and two prompting strategies (minimal vs. enriched with linguistic and psychological cues). Enriched
prompts reduce invalid outputs and improve class balance, but also introduce a systematic bias toward predicting trait presence. Performance varies substantially: Openness and Agreeableness are relatively easier to detect, while Extraversion and Neuroticism remain challenging. Although open-source models sometimes approach GPT-4 and prior
benchmarks, no configuration yields consistently reliable predictions in zero-shot binary settings. Moreover, aggregate metrics such as accuracy and macro-F1 mask significant asymmetries, with per-class recall offering clearer diagnostic value. These findings show that current out-of-the-box LLMs are not yet suitable for APPT, and that careful coordination of
prompt design, trait framing, and evaluation metrics is essential for interpretable results.
Read the full abstract ▾
All authors Francesco Di Cursi, Chiara Boldrini, Marco Conti, Andrea Passarella
Computation and Language
Accent Placement Models for Rigvedic Sanskrit Text
Akhil Rajeev P and Annarao Kulkarni
The Rigveda, among the oldest Indian texts in Vedic Sanskrit, employs a distinctive pitch-accent system : udātta, anudātta, svarita whose marks encode melodic and interpretive cues but are often absent from modern e-texts. This work develops a parallel corpus of accented-unaccented ślokas and conducts a controlled comparison of three strategies for automatic accent placement in Rigvedic verse: (i) full fine-tuning of ByT5, a byte-level
Transformer that operates directly on Unicode combining marks, (ii) a from-scratch BiLSTM-CRF sequence-labeling baseline, and (iii) LoRA-based parameter-efficient fine-tuning atop ByT5. Evaluation uses Word Error Rate (WER) and Character Error Rate (CER) for orthographic fidelity, plus a task-specific Diacritic Error Rate (DER) that isolates accent edits. Full ByT5 fine-tuning attains the lowest error across all metrics; LoRA offers strong efficiency-accuracy trade-offs, and BiLSTM-CRF serves as a transparent baseline. The study underscores practical requirements for accent restoration - Unicode-safe preprocessing, mark-aware tokenization, and evaluation that separates grapheme from accent errors - and positions heritage-
language technology as an emerging NLP area connecting computational modeling with philological and pedagogical aims. Results establish reproducible baselines for Rigvedic accent restoration and provide guidance for downstream tasks such as accent-aware OCR, ASR/chant synthesis, and digital scholarship.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Bharat Scene Text: A Novel Comprehensive Dataset and Benchmark for Indian Language Scene Text Understanding
Anik De, Abhirama Subramanyam Penamakuri, Rajeev Yadav et al. (+6 more)
Reading scene text, that is, text appearing in images, has numerous application areas, including assistive technology, search, and e-commerce. Although scene text recognition in English has advanced significantly and is often considered nearly a solved problem, Indian
language scene text recognition remains an open challenge. This is due to script diversity, non-standard fonts, and varying writing styles, and, more importantly, the lack of high-quality
datasets and open-source models. To address these gaps, we introduce the Bharat Scene Text
Dataset (BSTD) - a large-scale and comprehensive
benchmark for studying Indian
Language Scene Text Recognition. It comprises more than 100K words that span 11 Indian
languages and English, sourced from over 6,500 scene images captured across various linguistic regions of India. The
dataset is meticulously annotated and supports multiple scene text tasks, including: (i) Scene Text Detection, (ii) Script Identification, (iii) Cropped Word Recognition, and (iv) End-to-End Scene Text Recognition. We evaluated state-of-the-art models originally developed for English by adapting (fine-tuning) them for Indian
languages. Our results highlight the challenges and opportunities in Indian
language scene text recognition. We believe that this
dataset represents a significant step toward advancing research in this domain. All our models and data are open source.
Read the full abstract ▾
All authors Anik De, Abhirama Subramanyam Penamakuri, Rajeev Yadav, Aditya Rathore, Harshiv Shah, Devesh Sharma, Sagar Agarwal, Pravin Kumar, Anand Mishra
Computation and Language
Conveying Imagistic Thinking in Traditional Chinese Medicine Translation: A Prompt Engineering and LLM-Based Evaluation Framework
Jiatong Han
Traditional Chinese Medicine theory is built on imagistic thinking, in which medical principles and diagnostic and therapeutic logic are structured through metaphor and metonymy. However, existing English translations largely rely on literal rendering, making it difficult for target-
language readers to reconstruct the underlying conceptual networks and apply them in clinical practice. This study adopted a human-in-the-loop framework and selected four passages from the medical canon Huangdi Neijing that are fundamental in theory. Through
prompt-based cognitive scaffolding, DeepSeek V3.1 was guided to identify metaphor and metonymy in the source text and convey the theory in translation. In the evaluation stage, ChatGPT 5 Pro and Gemini 2.5 Pro were instructed by
prompts to simulate three types of real-world readers. Human translations, baseline model translations, and
prompt-adjusted translations were scored by the simulated readers across five cognitive dimensions, followed by structured interviews and Interpretative Phenomenological Analysis. Results show that the
prompt-adjusted LLM translations perform best across all five dimensions, with high cross-model and cross-role consistency. The interview themes reveal differences between human and machine translation, effective strategies for metaphor and metonymy transfer, and readers' cognitive preferences. This study provides a cognitive, efficient and replicable HITL methodological pathway for translation of ancient, concept-dense texts like TCM.
Read the full abstract ▾
Computation and Language
Standard Occupation Classifier – A Natural Language Processing Approach
Sidharth Rony and Jack Patman
Standard Occupational Classifiers (SOC) are systems used to categorize and classify different types of jobs and occupations based on their similarities in terms of job duties, skills, and qualifications. Integrating these facets with Big Data from job advertisement offers the prospect to investigate labour demand that is specific to various occupations. This project investigates the use of recent developments in natural
language processing to construct a classifier capable of assigning an occupation code to a given job advertisement. We develop various classifiers for both UK ONS SOC and US O*NET SOC, using different
Language Models. We find that an ensemble model, which combines Google BERT and a Neural Network classifier while considering job title, description, and skills, achieved the highest prediction accuracy. Specifically, the ensemble model exhibited a classification accuracy of up to 61% for the lower (or fourth) tier of SOC, and 72% for the third tier of SOC. This model could provide up to date, accurate information on the
evolution of the labour
market using job advertisements.
Read the full abstract ▾
Computation and Language
Decoding the Past: Explainable Machine Learning Models for Dating Historical Texts
Paulo J. N. Pinto, Armando J. Pinho and Diogo Pratas
Accurately dating historical texts is essential for organizing and interpreting cultural heritage collections. This article addresses temporal text classification using interpretable, feature-engineered tree-based machine learning models. We integrate five feature categories - compression-based, lexical structure, readability, neologism detection, and distance features - to predict the temporal origin of English texts spanning five centuries. Comparative analysis shows that these feature domains provide complementary temporal signals, with combined models outperforming any individual feature set. On a large-scale corpus, we achieve 76.7% accuracy for century-scale prediction and 26.1% for decade-scale classification, substantially above random baselines (20% and 2.3%). Under relaxed temporal precision, performance increases to 96.0% top-2 accuracy for centuries and 85.8% top-10 accuracy for decades. The final model exhibits strong ranking capabilities with AUCROC up to 94.8% and AUPRC up to 83.3%, and maintains controlled errors with mean absolute deviations of 27 years and 30 years, respectively. For authentication-style tasks, binary models around key thresholds (e.g., 1850-1900) reach 85-98% accuracy. Feature importance analysis identifies distance features and lexical structure as most informative, with compression-based features providing complementary signals. SHAP explainability reveals systematic linguistic
evolution patterns, with the 19th century emerging as a pivot point across feature domains. Cross-
dataset evaluation on Project Gutenberg highlights domain adaptation challenges, with accuracy dropping by 26.4 percentage points, yet the computational efficiency and interpretability of tree-based models still offer a scalable, explainable alternative to neural architectures.
Read the full abstract ▾
Computation and Language
Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responses
Dong Nguyen and Laura Rosseel
Spelling variation (e.g. funnnn vs. fun) can influence the social perception of texts and their writers: we often have various associations with different forms of writing (is the text informal? does the writer seem young?). In this study, we focus on the social perception of spelling variation in online writing in English and study to what extent this perception is aligned between humans and large
language models (LLMs). Building on sociolinguistic methodology, we compare LLM and human ratings on three key social attributes of spelling variation (formality, carefulness, age). We find generally strong correlations in the ratings between humans and LLMs. However, notable differences emerge when we analyze the distribution of ratings and when comparing between different types of spelling variation.
Read the full abstract ▾
Computation and Language
ShoppingComp: Are LLMs Really Ready for Your Shopping Cart?
Huaixiao Tou, Ying Zeng, Yuemeng Li et al. (+6 more)
We present ShoppingComp, a challenging real-world
benchmark for comprehensively evaluating LLM-powered shopping
agents on three core capabilities: precise product
retrieval, expert-level report generation, and safety critical decision making. Unlike prior e-commerce
benchmarks, ShoppingComp introduces difficult product discovery queries with many constraints, while guaranteeing open-world products and enabling easy verification of
agent outputs. The
benchmark comprises 145 instances and 558 scenarios, curated by 35 experts to reflect authentic shopping needs. Results reveal stark limitations of current LLMs: even state-of-the-art models achieve low performance (e.g., 17.76% for GPT-5.2, 15.82% for Gemini-3-Pro).Error analysis reflects limitations in core
agent competencies, including information grounding in open-world environments, reliable verification of multi-constraint requirements, consistent
reasoning over noisy and conflicting evidence, and risk-aware decision making. By exposing these capability gaps, ShoppingComp characterizes the trust threshold that AI systems must cross before they can be proactively trusted for reliable real-world decision making. Our code and
dataset are available at https://github.com/ByteDance-BandAI/ShoppingComp.
Read the full abstract ▾
All authors Huaixiao Tou, Ying Zeng, Yuemeng Li, Cong Ma, Muzhi Li, Minghao Li, Weijie Yuan, He Zhang, Kai Jia
Computation and Language
Pooling Attention: Evaluating Pretrained Transformer Embeddings for Deception Classification
Sumit Mamtani and Abhijeet Bhure
This paper investigates fake news detection as a downstream evaluation of
Transformer representations, benchmarking encoder-only and decoder-only pre-trained models (BERT, GPT-2,
Transformer-XL) as frozen embedders paired with lightweight classifiers. Through controlled preprocessing comparing pooling versus padding and neural versus linear heads, results demonstrate that contextual self-
attention encodings consistently transfer effectively. BERT
embeddings combined with logistic regression outperform neural baselines on LIAR
dataset splits, while analyses of sequence length and aggregation reveal robustness to truncation and advantages from simple max or average pooling. This work positions
attention-based
token encoders as robust, architecture-centric foundations for veracity tasks, isolating
Transformer contributions from classifier complexity.
Read the full abstract ▾
Computation and Language
Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match
Jinze Li, Yixing Xu, Guanchen Li et al. (+6 more)
Large
language models (LLMs) achieve strong performance across diverse tasks but suffer from high inference latency due to their autoregressive generation. Speculative Decoding (SPD) mitigates this issue by verifying candidate
tokens in parallel from a smaller draft model, yet its strict exact-match verification discards many semantically valid continuations. Moreover, existing training-based SPD methods often suffer from performance degradation on out-of-distribution (OOD) tasks. To this end, we propose Training-Free Loosely Speculative Decoding (FLy), a novel method that loosens the rigid verification criterion by leveraging the target model's self-corrective behavior to judge whether a draft-target mismatch remains semantically valid. FLy introduces a two-tier mechanism: an
entropy-level gate that identifies whether the current
token allows multiple plausible alternatives or is nearly deterministic, and a
token-level deferred window that distinguishes genuine errors from differently worded yet semantically correct variants. To further reduce latency, we design a multi-level acceleration strategy that accelerates not only the target model but also the drafter itself. Owing to its training-free design, FLy composes seamlessly with arbitrary draft-target pairs and generalizes across models and domains without hyperparameter re-tuning. Experiments show that FLy preserves more than 99% of the target model's accuracy while achieving an average 2.81x speedup on Llama-3.1-70B-Instruct and 5.07x speedup on the 405B variant. Notably, on out-of-domain
datasets, our method remains highly effective and outperforms the training-based method EAGLE-3 by 1.62x.
Read the full abstract ▾
All authors Jinze Li, Yixing Xu, Guanchen Li, Shuo Yang, Jinfeng Xu, Xuanwu Yin, Dong Li, Edith C.H.Ngai, Emad Barsoum
Computation and Language
Visual Puns from Idioms: An Iterative LLM-T2IM-MLLM Framework
Kelaiti Xiao, Liang Yang, Dongyu Zhang et al. (+2 more)
We study idiom-based visual puns–images that align an idiom's literal and figurative meanings–and present an iterative framework that coordinates a large
language model (LLM), a text-to-image model (T2IM), and a
multimodal LLM (MLLM) for automatic generation and evaluation. Given an idiom, the system iteratively (i) generates detailed visual
prompts, (ii) synthesizes an image, (iii) infers the idiom from the image, and (iv) refines the
prompt until recognition succeeds or a step limit is reached. Using 1,000 idioms as inputs, we synthesize a corresponding
dataset of visual pun images with paired
prompts, enabling benchmarking of both generation and understanding. Experiments across 10 LLMs, 10 MLLMs, and one T2IM (Qwen-Image) show that MLLM choice is the primary performance driver: GPT achieves the highest accuracies, Gemini follows, and the best open-source MLLM (Gemma) is competitive with some closed models. On the LLM side, Claude attains the strongest average performance for
prompt generation.
Read the full abstract ▾
All authors Kelaiti Xiao, Liang Yang, Dongyu Zhang, Paerhati Tulajiang, Hongfei Lin
Computation and Language · Cross-listed from cs.CV
Artwork Interpretation with Vision Language Models: A Case Study on Emotions and Emotion Symbols
Sebastian Padó and Kerstin Thomas
Emotions are a fundamental aspect of artistic expression. Due to their abstract nature, there is a broad spectrum of
emotion realization in artworks. These are subject to historical change and their analysis requires expertise in art history. In this article, we investigate which aspects of emotional expression can be detected by current (2025)
vision language models (VLMs). We present a case study of three VLMs (Llava-Llama and two Qwen models) in which we ask these models four sets of questions of increasing complexity about artworks (general content, emotional content, expression of
emotions, and
emotion symbols) and carry out a qualitative expert evaluation. We find that the VLMs recognize the content of the images surprisingly well and often also which
emotions they depict and how they are expressed. The models perform best for concrete images but fail for highly abstract or highly symbolic images. Reliable recognition of symbols remains fundamentally difficult. Furthermore, the models continue to exhibit the well-known LLM weakness of providing inconsistent answers to related questions.
Read the full abstract ▾
Computation and Language
Language-conditioned world model improves policy generalization by reading environmental descriptions
Anh Nguyen and Stefan Lee
To interact effectively with humans in the real world, it is important for
agents to understand
language that describes the dynamics of the environment–that is, how the environment behaves–rather than just task instructions specifying "what to do". Understanding this dynamics-descriptive
language is important for human-
agent interaction and
agent behavior. Recent work address this problem using a model-based approach:
language is incorporated into a world model, which is then used to learn a behavior policy. However, these existing methods either do not demonstrate policy generalization to unseen games or rely on limiting assumptions. For instance, assuming that the latency induced by inference-time planning is tolerable for the target task or expert demonstrations are available. Expanding on this line of research, we focus on improving policy generalization from a
language-conditioned world model while dropping these assumptions. We propose a model-based reinforcement learning approach, where a
language-conditioned world model is trained through interaction with the environment, and a policy is learned from this model–without planning or expert demonstrations. Our method proposes
Language-aware Encoder for Dreamer World Model (LED-WM) built on top of DreamerV3. LED-WM features an observation encoder that uses an
attention mechanism to explicitly ground
language descriptions to entities in the observation. We show that policies trained with LED-WM generalize more effectively to unseen games described by novel dynamics and
language compared to other baselines in several settings in two environments: MESSENGER and MESSENGER-WM.To highlight how the policy can leverage the trained world model before real-world deployment, we demonstrate the policy can be improved through fine-tuning on synthetic test trajectories generated by the world model.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
ORION: Teaching Language Models to Reason Efficiently in the Language of Thought
Kumar Tanmay, Kriti Aggarwal, Paul Pu Liang et al. (+1 more)
Large
Reasoning Models (LRMs) achieve strong performance in mathematics, code generation, and task planning, but their reliance on long chains of verbose "thinking"
tokens leads to high latency, redundancy, and incoherent
reasoning paths. Inspired by the
Language of Thought Hypothesis, which posits that human
reasoning operates over a symbolic, compositional mental
language called Mentalese, we introduce a framework that trains models to reason in a similarly compact style. Mentalese encodes abstract
reasoning as ultra-compressed, structured
tokens, enabling models to solve complex problems with far fewer steps. To improve both efficiency and accuracy, we propose SHORTER LENGTH PREFERENCE OPTIMIZATION (SLPO), a reinforcement learning method that rewards concise solutions that stay correct, while still allowing longer
reasoning when needed. Applied to Mentalese-aligned models, SLPO yields significantly higher compression rates by enabling concise
reasoning that preserves the benefits of detailed thinking without the computational overhead. Across
benchmarks including AIME 2024 and 2025, MinervaMath, OlympiadBench, Math500, and AMC, our ORION models produce
reasoning traces with 4-16x fewer
tokens, achieve up to 5x lower inference latency, and reduce training costs by 7-9x relative to the DeepSeek R1 Distilled model, while maintaining 90-98% of its accuracy. ORION also surpasses Claude and ChatGPT-4o by up to 5% in accuracy while maintaining 2x compression. These results show that Mentalese-style compressed
reasoning offers a step toward human-like cognitive efficiency, enabling real-time, cost-effective
reasoning without sacrificing accuracy.
Read the full abstract ▾
All authors Kumar Tanmay, Kriti Aggarwal, Paul Pu Liang, Subhabrata Mukherjee
Computation and Language
FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writing
Jingheng Ye, Shen Wang, Jiaqi Chen et al. (+9 more)
Large
Language Models (LLMs) have transformed artificial intelligence, offering profound opportunities for educational applications. However, their ability to provide fine-grained educational feedback for K-12 English writing remains underexplored. In this paper, we challenge the error analysis and pedagogical skills of LLMs by introducing the problem of Fine-grained Error Analysis for English Learners and present the Fine-grained Error ANalysis for English Learners (FEANEL)
Benchmark. The
benchmark comprises 1,000 essays written by elementary and secondary school students, and a well-developed English writing error taxonomy. Each error is annotated by
language education experts and categorized by type, severity, and explanatory feedback, using a part-of-
speech-based taxonomy they co-developed. We evaluate state-of-the-art LLMs on the FEANEL
Benchmark to explore their error analysis and pedagogical abilities. Experimental results reveal significant gaps in current LLMs' ability to perform fine-grained error analysis, highlighting the need for advancements in particular methods for educational applications.
Read the full abstract ▾
All authors Jingheng Ye, Shen Wang, Jiaqi Chen, Hebin Wang, Deqing Zou, Yanyu Zhu, Jiwei Tang, Hai-Tao Zheng, Ruitong Liu, Haoyang Li, Yanfeng Wang, Qingsong Wen
Computation and Language
JBE-QA: Japanese Bar Exam QA Dataset for Assessing Legal Domain Knowledge
Zhihan Cao, Fumihito Nishino, Hiroaki Yamada et al. (+3 more)
We introduce JBE-QA, a Japanese Bar Exam Question-Answering
dataset to evaluate large
language models' legal knowledge. Derived from the multiple-choice (tanto-shiki) section of the Japanese bar exam (2015-2024), JBE-QA provides the first comprehensive
benchmark for Japanese legal-domain evaluation of LLMs. It covers the Civil Code, the Penal Code, and the Constitution, extending beyond the Civil Code focus of prior Japanese resources. Each question is decomposed into independent true/false judgments with structured contextual fields. The
dataset contains 3,464 items with balanced labels. We evaluate 26 LLMs, including proprietary, open-weight, Japanese-specialised, and
reasoning models. Our results show that proprietary models with
reasoning enabled perform best, and the Constitution questions are generally easier than the Civil Code or the Penal Code questions.
Read the full abstract ▾
All authors Zhihan Cao, Fumihito Nishino, Hiroaki Yamada, Nguyen Ha Thanh, Yusuke Miyao, Ken Satoh
Computation and Language
RAG System for Supporting Japanese Litigation Procedures: Faithful Response Generation Complying with Legal Norms
Yuya Ishihara, Atsushi Keyaki, Hiroaki Yamada et al. (+2 more)
This study discusses the essential components that a
Retrieval-Augmented Generation (RAG)-based LLM system should possess in order to support Japanese medical litigation procedures complying with legal norms. In litigation, expert commissioners, such as physicians, architects, accountants, and engineers, provide specialized knowledge to help judges clarify points of dispute. When considering the substitution of these expert roles with a RAG-based LLM system, the constraint of strict adherence to legal norms is imposed. Specifically, three requirements arise: (1) the
retrieval module must retrieve appropriate external knowledge relevant to the disputed issues in accordance with the principle prohibiting the use of private knowledge, (2) the responses generated must originate from the context provided by the RAG and remain faithful to that context, and (3) the
retrieval module must reference external knowledge with appropriate timestamps corresponding to the issues at hand. This paper discusses the design of a RAG-based LLM system that satisfies these requirements.
Read the full abstract ▾
All authors Yuya Ishihara, Atsushi Keyaki, Hiroaki Yamada, Ryutaro Ohara, Mihoko Sumida