Computation and Language
SAGE: A Strategy-Aware Graph-Enhanced Generation Framework For Online Counseling
Eliya Naomi Aharon, Meytal Grimland, Avi Segal et al. (+4 more)
Effective mental health counseling is a complex, theory-driven process requiring the simultaneous integration of psychological frameworks, real-time distress signals, and strategic intervention planning. This level of clinical
reasoning is critical for safety and therapeutic effectiveness but is often missing in general-purpose Large
Language Models (LLMs). We introduce SAGE (Strategy-Aware Graph-Enhanced), a novel framework designed to bridge the gap between structured clinical knowledge and generative AI. SAGE constructs a heterogeneous graph that unifies conversational dynamics with a psychologically grounded layer, explicitly anchoring interactions in a theory-driven lexicon. Our architecture first employs a Next Strategy Classifier to identify the optimal therapeutic intervention. Subsequently, a Graph-Aware
Attention mechanism projects graph-derived structural signals into soft
prompts, conditioning the LLM to generate responses that maintain clinical depth. Validated through both automated metrics and expert human evaluation, SAGE outperforms baselines in strategy prediction and recommended response quality. By providing actionable intervention recommendations, SAGE serves as a cutting-edge decision-support tool designed to augment human expertise in high-stakes crisis counseling.
Read the full abstract ▾
All authors Eliya Naomi Aharon, Meytal Grimland, Avi Segal, Loona Ben Dayan, Inbar Shenfeld, Yossi Levi Belz, Kobi Gal
Computation and Language
Jinze Li, Yang Zhang, Xin Yang et al. (+5 more)
Autonomous LLM
agents increasingly operate in long-horizon, interactive settings where success depends on reusing experience accumulated over extended histories. However, existing
agent memory systems are fundamentally constrained by text-context budgets: storing or revisiting raw trajectories is prohibitively
token-expensive, while summarization and text-only
retrieval trade
token savings for information loss and fragmented evidence. To address this limitation, we propose Optical Context
Retrieval Memory (OCR-
Memory), a
memory framework that leverages the visual modality as a high-density representation of
agent experience, enabling retention of arbitrarily long histories with minimal
prompt overhead at
retrieval time. Specifically, OCR-
Memory renders historical trajectories into images annotated with unique visual identifiers. OCR-
Memory retrieves stored experience via a locate-and-transcribe paradigm that selects relevant regions through visual anchors and retrieves the corresponding verbatim text, avoiding free-form generation and reducing
hallucination. Experiments on long-horizon
agent benchmarks show consistent gains under strict context limits, demonstrating that optical encoding increases effective
memory capacity while preserving faithful evidence recovery.
Read the full abstract ▾
All authors Jinze Li, Yang Zhang, Xin Yang, Jiayi Qu, Jinfeng Xu, Shuo Yang, Junhua Ding, Edith Cheuk-Han Ngai
Computation and Language
Zero-Shot to Full-Resource: Cross-lingual Transfer Strategies for Aspect-Based Sentiment Analysis
Jakob Fehle, Nils Constantin Hellwig, Udo Kruschwitz et al. (+1 more)
Aspect-based Sentiment Analysis (ABSA) extracts fine-grained opinions toward specific aspects within text but remains largely English-focused despite major advances in
transformer-based and instruction-tuned models. This work presents a
multilingual evaluation of state-of-the-art ABSA approaches across seven
languages (English, German, French, Dutch, Russian, Spanish, and Czech) and four subtasks (ACD, ACSA, TASD, ASQP). We systematically compare different
transformer architectures under zero-resource, data-only, and full-resource settings, using cross-lingual transfer, code-switching and machine translation. Fine-tuned Large
Language Models (LLMs) achieve the highest overall scores, particularly in complex generative tasks, while few-shot counterparts approach this performance in simpler setups, where smaller encoder models also remain competitive. Cross-lingual training on multiple non-target
languages yields the strongest transfer for fine-tuned LLMs, while smaller encoder or seq-to-seq models benefit most from code-switching, highlighting architecture-specific strategies for
multilingual ABSA. We further contribute two new German
datasets, an adapted GERestaurant and the first German ASQP
dataset (GERest), to encourage
multilingual ABSA research beyond English.
Read the full abstract ▾
All authors Jakob Fehle, Nils Constantin Hellwig, Udo Kruschwitz, Christian Wolff
Computation and Language
Translating Under Pressure: Domain-Aware LLMs for Crisis Communication
Antonio Castaldo, Maria Carmen Staiano, Johanna Monti et al. (+2 more)
Timely and reliable
multilingual communication is critical during natural and human-induced disasters, but developing effective solutions for crisis communication is limited by the scarcity of curated parallel data. We propose a domain-adaptive pipeline that expands a small reference corpus, by retrieving and filtering data from general corpora. We use the resulting
dataset to fine-tune a small
language model for crisis-domain translation and then apply preference optimization to bias outputs toward CEFR A2-level English. Automatic and human evaluation shows that this approach improves readability, while maintaining strong adequacy. Our results indicate that simplified English, combined with domain adaptation, can function as a practical lingua franca for emergency communication when full
multilingual coverage is not feasible.
Read the full abstract ▾
All authors Antonio Castaldo, Maria Carmen Staiano, Johanna Monti, Sheila Castilho, Francesca Chiusaroli
Computation and Language
Multimodal LLMs are not all you need for Pediatric Speech Language Pathology
Darren Fürst, Sebastian Steindl and Ulrich Schäfer
Speech Sound Disorders (SSD) affect roughly five percent of children, yet
speech-
language pathologists face severe staffing shortages and unmanageable caseloads. We test a hierarchical approach to SSD classification on the granular multi-task SLPHelmUltraSuitePlus
benchmark. We propose a cascading approach from binary classification to type, and symptom classification. By fine-tuning
Speech Representation Models (SRM), and using targeted data augmentation we mitigate biases found by previous works, and improve upon all clinical tasks in the
benchmark. We also treat Automatic
Speech Recognition (ASR) with our data augmentation approach. Our results demonstrate that SRM consistently outperform the LLM-based state-of-the-art across all evaluated tasks by a large margin. We publish our models and code to foster future research.
Read the full abstract ▾
Computation and Language
TLPO: Token-Level Policy Optimization for Mitigating Language Confusion in Large Language Models
Jinho Choo, JunSeung Lee, Jimyeong Kim et al. (+3 more)
Large
language models (LLMs) demonstrate strong
multilingual capabilities, yet often fail to consistently generate responses in the intended
language, exhibiting a phenomenon known as
language confusion. Prior mitigation approaches based on sequence-level fine-tuning, such as DPO, ORPO, and GRPO, operate at the level of entire responses and can lead to unintended degradation of general model capabilities, motivating the need for more fine-grained alternatives. To address this, we introduce
Token-Level Policy Optimization (TLPO), a fine-tuning framework designed to mitigate
language confusion through localized,
token-level updates. TLPO identifies error-prone positions, explores alternative candidate
tokens, and updates the policy using a tailored objective to suppress error-inducing outputs at a granular level. This selective intervention enables effective mitigation of
language confusion without compromising the model's general abilities. Experiments on multiple
multilingual LLMs across diverse
languages demonstrate that TLPO significantly outperforms baselines in improving
language consistency while preserving downstream task accuracy.
Read the full abstract ▾
All authors Jinho Choo, JunSeung Lee, Jimyeong Kim, Yeeho Song, S. K. Hong, Yeong-Dae Kwon
Computation and Language
Text-Utilization for Encoder-dominated Speech Recognition Models
Albert Zeyer, Tim Posielek, Ralf Schlüter et al. (+1 more)
This paper investigates efficient methods for utilizing text-only data to improve
speech recognition, focusing on encoder-dominated models that facilitate faster recognition. We provide a comprehensive comparison of techniques to integrate text-only data, including modality matching and dynamic downsampling to reach text-level representations within the encoder. Our experiments on the LibriSpeech corpus show that a larger encoder with a smaller decoder can equal or surpass the performance of architectures with larger decoders. We demonstrate that simple configurations, such as random duration models, are often more effective than complex alternatives, significantly simplifying the training pipeline. All code and recipes are made publicly available.
Read the full abstract ▾
All authors Albert Zeyer, Tim Posielek, Ralf Schlüter, Hermann Ney
Computation and Language
SafeReview: Defending LLM-based Review Systems Against Adversarial Hidden Prompts
Yuan Xin, Yixuan Weng, Minjun Zhu et al. (+5 more)
As Large
Language Models (LLMs) are increasingly integrated into academic peer review, their vulnerability to
adversarial hidden
prompts, i.e.,
adversarial instructions embedded in submissions to manipulate outcomes, poses a critical threat to scholarly integrity. We propose SafeReview, a co-evolutionary
adversarial training framework for defending LLM-based peer review systems against such attacks. SafeReview jointly trains a Generator model to create sophisticated attack
prompts and a Defender model to preserve review integrity under
adversarial manipulation. The Generator is optimized to produce increasingly effective
prompt injections, while the Defender is strengthened through preference-based training to maintain consistent reviews between clean and attacked submissions. Experimental results show that SafeReview improves robustness against adaptive
prompt injection attacks, better preserves paper ranking under attack, and generalizes across attacker architectures compared with static defenses. These results demonstrate the potential of co-evolutionary training as a foundation for securing LLM-assisted peer review.
Read the full abstract ▾
All authors Yuan Xin, Yixuan Weng, Minjun Zhu, Ying Ling, Chengwei Qin, Michael Backes, Yue Zhang, Linyi Yang
Computation and Language
Tree-of-Text: A Tree-based Prompting Framework for Table-to-Text Generation in the Sports Domain
Shang-Hsuan Chiang, Tsan-Tsung Yang, An-Zi Yen et al. (+1 more)
Generating sports game reports from structured tables is a complex table-to-text task that demands both precise data interpretation and fluent narrative generation. Traditional model-based approaches require large, annotated
datasets, while
prompt-based methods using large
language models (LLMs) often struggle with
hallucination due to weak table comprehension. To overcome these challenges, we propose Tree-of-Text, a tree-structured prompting framework that guides LLMs through a three-stage generation process: (1) Content Planning, where relevant operations and arguments are selected from the input tables; (2) Operation Execution, which breaks down large tables into manageable sub-tables; and (3) Content Generation, where short textual outputs are merged and rewritten into a cohesive report. Experiments show that our method outperforms existing methods on ShuttleSet+, leads in RG and CO metrics on RotoWire-FG, and excels in CS and CO on MLB with roughly 40% of the time and cost of Chain-of-Table. These results demonstrate the effectiveness and efficiency of Tree-of-Text and suggest a promising direction for
prompt-based table-to-text generation in the sports domain.
Read the full abstract ▾
All authors Shang-Hsuan Chiang, Tsan-Tsung Yang, An-Zi Yen, Wen-Chih Peng
Computation and Language
StarDrinks: An English and Korean Test Set for SLU Evaluation in a Drink Ordering Scenario
Marcely Zanon Boito, Caroline Brun, Inyoung Kim et al. (+5 more)
LLMs and
speech assistants are increasingly used for task-oriented interactions, yet their evaluation often relies on controlled scenarios that fail to capture the variability and complexity of real user requests. Drink ordering, for example, involves diverse named entities, drink types, sizes, customizations, and brand-specific terminology, as well as spontaneous
speech phenomena such as hesitations and self-corrections. To address this gap, we introduce StarDrinks, a test set in English and Korean containing
speech utterances features, transcriptions, and annotated slots. Our
dataset supports
speech-to-slots SLU, transcription-to-slots NLU, and
speech-to-transcription ASR evaluation, providing a realistic
benchmark for model robustness and generalization in a linguistically rich, real-world task.
Read the full abstract ▾
All authors Marcely Zanon Boito, Caroline Brun, Inyoung Kim, Denys Proux, Salah Ait-Mokhtar, Nikolaos Lagos, Jean-Luc Meunier, Ioan Calapodescu
Computation and Language
Theory-Grounded Evaluation Exposes the Authorship Gap in LLM Personalization
Yash Ganpat Sawant
Stylistic personalization - making LLMs write in a specific individual's style, rather than merely adapting to task preferences - lacks evaluation grounded in authorship science. We show that grounding evaluation in authorship verification theory transforms what
benchmarks can measure. Drawing on three measurement traditions - LUAR (a trained authorship verification model), an LLM-as-judge with decoupled trait matching, and classical function-word stylometrics - we evaluate four inference-time personalization methods across 50 authors and 1,000 generations. The theory-grounded metric (LUAR) provides what ad hoc alternatives cannot: calibrated baselines (human ceiling 0.756, cross-author floor 0.626) that give scores absolute meaning. All methods score below this floor (0.484-0.508), exposing an authorship gap invisible to uncalibrated metrics. The three metrics produce near-zero pairwise correlations (|r| < 0.07), confirming that without theoretical grounding, metric choice determines conclusions - an LLM judge declares a clear winner while LUAR finds no meaningful differentiation. These findings demonstrate the theory-
benchmark cycle in action: authorship theory exposes evaluation failures that ad hoc
benchmarks miss.
Read the full abstract ▾
Computation and Language
Naamah: A Large Scale Synthetic Sanskrit NER Corpus via DBpedia Seeding and LLM Generation
Akhil Rajeev P and Annarao Kulkarni
The digitisation of classical Sanskrit literature is impeded by a scarcity of annotated resources, particularly for Named Entity Recognition. While recent methodologies utilise generic Large
Language Models (LLMs) for data augmentation, these approaches remain prone to error and often lack the
reasoning depth required for classical grammar. In this work, we introduce Naamah, a high quality silver standard Sanskrit NER
dataset comprising 102,942 sentences. We propose a methodology that combines entity extraction from DBpedia with the generative capabilities of a 24B parameter hybrid
reasoning model to create grammatically natural and synthetically diverse training data. We utilize this
dataset to
benchmark two
transformer architectures: the massive
multilingual XLM RoBERTa and the parameter efficient IndicBERTv2.
Read the full abstract ▾
Computation and Language
EmoTransCap: Dataset and Pipeline for Emotion Transition-Aware Speech Captioning in Discourses
Shuhao Xu, Yifan Hu, Jingjing Wu et al. (+3 more)
Emotion perception and adaptive expression are fundamental capabilities in human-
agent interaction. While recent advances in
speech emotion captioning (SEC) have improved fine-grained emotional modeling, existing systems remain limited to static, single-
emotion characterization within isolated sentences, neglecting dynamic emotional transitions at the discourse level. To address this gap, we propose
Emotion Transition-Aware
Speech Captioning (EmoTransCap), a paradigm that integrates temporal
emotion dynamics with discourse-level
speech description. To construct a
dataset rich in
emotion transitions while enabling scalable expansion, we design an automated pipeline for
dataset creation. This is the first large-scale
dataset explicitly designed to capture discourse-level
emotion transitions. To generate semantically rich descriptions, we incorporate acoustic attributes and temporal cues from discourse-level
speech. Our Multi-Task
Emotion Transition Recognition (MTETR) model performs joint
emotion transition detection and diarization. Leveraging the semantic analysis capabilities of LLMs, we produce two annotation versions: descriptive and instruction-oriented. These data and annotations offer a valuable resource for advancing
emotion perception and emotional expressiveness. The
dataset enables
speech captions that capture emotional transitions, facilitating temporal-dynamic and fine-grained
emotion understanding. We also introduce a controllable, transition-aware emotional
speech synthesis system at the discourse level, enhancing anthropomorphic emotional expressiveness and supporting emotionally intelligent conversational
agents.
Read the full abstract ▾
All authors Shuhao Xu, Yifan Hu, Jingjing Wu, Zhihao Du, Zheng Lian, Rui Liu
Computation and Language
When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding?
Tianyu Liu, Yuhao Shen, Xinyi Hu et al. (+8 more)
Speculative decoding accelerates LLM inference, but SOTA hidden-state-based drafters suffer from long-range decay: draft accuracy degrades as the speculative step increases. Existing work attributes this decay to train-inference mismatch and proposes test-time training (TTT) as a remedy, yet we observe that long-range decay persists even in TTT-trained drafters. We revisit long-range decay from the perspective of context information preservation. In hidden-state reuse, we argue the target hidden state acts as a biased context compression: it aggregates historical
token information according to the
attention query at the current position, yielding a compact representation optimized for immediate next-
token prediction. This compression can suppress information less relevant to the current query but important for later speculative steps. In contrast, the target model's KV cache serves as an explicit context, retaining the complete set of
token-wise KV representations. We therefore posit the KV-Reuse Hypothesis: allowing the draft model to reuse the target KV cache can provide richer signals for long-horizon drafting. To test this hypothesis, we introduce KVShot, a diagnostic framework that compares three reuse paradigms: hidden-only, KV-only, and hybrid. Extensive evaluations on Qwen3-8B show that KV-Reuse improves long-range acceptance, although end-to-end speedups remain marginal under current training pipelines. Our analysis identifies two key structural bottlenecks: shallow drafters struggle to estimate target queries accurately, and draft-side KV projections receive sparse gradient signals. These findings suggest that realizing the full potential of KV-aware decoding requires moving beyond TTT toward block-wise training paradigms. By exposing these bottlenecks, KVShot provides a foundational diagnostic testbed and a clear roadmap for designing next-generation inference architectures.
Read the full abstract ▾
All authors Tianyu Liu, Yuhao Shen, Xinyi Hu, Baolin Zhang, Hengxin Zhang, Jun Dai, Jun Zhang, Shuang Ge, Lei Chen, Yue Li, MingCheng Wan
Computation and Language
Benchmarking Complex Multimodal Document Processing Pipelines: A Unified Evaluation Framework for Enterprise AI
Saurabh K. Singh and Sachin Raj
Most enterprise document AI today is a pipeline. Parse, index, retrieve, generate. Each of those stages has been studied to death on its own – what's still hard is evaluating the system as a whole. We built EnterpriseDocBench to take a swing at it: parsing fidelity, indexing efficiency,
retrieval relevance, and generation groundedness, all on the same corpus. The corpus is built from public, permissively licensed documents across six enterprise domains (five represented in the current pilot). We ran three pipelines through it – BM25, dense
embedding, and a hybrid – all with the same GPT-5 generator. The headline numbers: hybrid
retrieval narrowly beats BM25 (nDCG@5 of 0.92 vs. 0.91), and both beat dense
embedding (0.83).
Hallucination doesn't grow monotonically with document length – short documents and very long ones both hallucinate more than medium ones (28.1% and 23.8% vs. 9.2%). Cross-stage correlations are very weak: parsing->
retrieval r=0.14, parsing->generation r=0.17,
retrieval->generation 0.02. If quality were cascading the way most of us assume, those numbers would be much higher; they aren't. Design caveats are real (parsing fixed, generator shared, automated proxy metrics) and we don't oversell the result. One result that genuinely surprised us: factual accuracy on stated claims is 85.5%, but answer completeness averages 0.40. The system is right when it answers – it just leaves things out. That gap matters more for real deployments than the headline accuracy number does. We also describe three reference architectures (ColPali, ColQwen2, agentic complexity-based routing) which are not yet integrated end-to-end. Framework, metrics, baselines, and collection scripts will be released open-source on acceptance.
Read the full abstract ▾
Computation and Language
SG-UniBuc-NLP at SemEval-2026 Task 6: Multi-Head RoBERTa with Chunking for Long-Context Evasion Detection
Gabriel Stefan and Sergiu Nisioi
We describe our system for SemEval-2026 Task 6 (CLARITY: Unmasking Political Question Evasions), which classifies English political interview responses by coarse-grained clarity (3-way) and fine-grained evasion strategy (9-way). Since responses frequently exceed the 512-
token limit of standard
Transformer encoders, we apply an overlapping sliding-window chunking strategy with element-wise Max-Pooling aggregation over chunk representations. A shared RoBERTa-large encoder supplies two task-specific heads trained jointly via a multi-task objective, with inference-time ensembling over 7-fold stratified cross-validation. Our system achieves a Macro-F1 of 0.80 on Subtask 1 and 0.51 on Subtask 2, ranking 11th in both subtasks.
Read the full abstract ▾
Computation and Language
Text Style Transfer with Machine Translation for Graphic Designs
Deergh Singh Budhauria, Sanyam Jain, Rishav Agarwal et al. (+1 more)
Globalization of graphic designs such as those used in marketing materials and magazines is increasingly important for communication to broad audiences. To accomplish this, the textual content in the graphic designs needs to be accurately translated and have the text styling preserved in order to fit visually into the design. Preserving text styling requires high accuracy word
alignment between the original and the translated text. The problem of word
alignment between source and translated text is long known. The industry standards for extracting word
alignments are defined by Giza++ and
attention probabilities from neural machine translation (NMT) models. In this paper, we explore three new methods to tackle the word
alignment problem for transferring text styles from the source to the translated text. The proposed methods are developed on top of commercially available NMT and LLM translation technologies. They include: NMT with custom input and output tags for text styling; LLM with custom input and output tags; a hybrid with NMT for translation followed by an LLM with use of unigram mappings. To analyze the performance of these solutions, their
alignment results are compared with the results of an
attention head approach to gauge their usability in graphic design applications. Interestingly, the
attention head strong baseline proves more accurate than the LLM or NMT approach and on par with the hybrid NMT+LLM approach.
Read the full abstract ▾
All authors Deergh Singh Budhauria, Sanyam Jain, Rishav Agarwal, Tracy King
Computation and Language
Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens
Zhenyu Zhao, Sander Land, Daniel M. Bikel et al. (+1 more)
Reasoning in Large
Language Models incurs significant inference-time compute, yet the
token-level information structure of
reasoning traces remains underexplored. We observe that
reasoning tokens split into two functional types: low-
entropy structural
tokens (recurring phrases that scaffold the
reasoning process) and higher-
entropy organic
tokens (problem-specific content that drives toward a solution). This asymmetry motivates a simple, model-agnostic compression pipeline: apply cross-word BPE merges on a model's own
reasoning traces to derive supertokens that capture frequent structural patterns, then teach the model to adopt them via supervised fine-tuning. Across three model families and five mathematical
reasoning benchmarks, our approach shortens
reasoning traces by 8.1% on average; under a TOST equivalence analysis at a +/- 2pp margin, accuracy is equivalent or inconclusive on 13/15 model –
benchmark cells (2 pass equivalence, 11 inconclusive, predominantly AIME at N=30, with non-equivalent degradation on 2/15
cells (DeepSeek-R1-Distill-Llama-70B on MATH-500 and OlympiadBench). Beyond compression, learned supertokens often align with interpretable
reasoning moves such as backtracking, verification, and strategy shifts. This enables a compact structural analysis of
reasoning traces: correct traces show more recovery and verification patterns, while incorrect traces show more repeated hedging and unresolved counterarguments. We release the full pipeline as open-source code.
Read the full abstract ▾
All authors Zhenyu Zhao, Sander Land, Daniel M. Bikel, Waseem Alshikh
Computation and Language
A Dual-Task Paradigm to Investigate Sentence Comprehension Strategies in Language Models
Rei Emura and Saku Sugawara
Language models (LMs) behave more like humans when their cognitive resources are restricted, particularly in predicting sentence processing costs such as reading times. However, it remains unclear whether such constraints similarly affect sentence comprehension strategies. Besides, existing methods do not directly target the balance between
memory storage and sentence processing, which is central to human working
memory. To address this issue, we propose a dual-task paradigm that combines an arithmetic computation task with a sentence comprehension task, such as "The 2 cocktail + blended 3 =..." Our experiments show that under dual-task conditions, GPT-4o, o3-mini, and o4-mini shift toward plausibility-based comprehension, mirroring humans' rational inference. Specifically, these models show a greater accuracy gap between plausible sentences (e.g., "The cocktail was blended by the bartender") and implausible sentences (e.g., "The bartender was blended by the cocktail") in the dual-task condition compared to the single-task conditions. These findings suggest that constraints on the balance between
memory and processing resources promote rational inference in LMs. More broadly, they support the view that human-like sentence comprehension fundamentally arises from the allocation of limited cognitive resources.
Read the full abstract ▾
Computation and Language · Cross-listed from eess.AS
The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluation
Yun-Shao Tsai, Yi-Cheng Lin, Huang-Cheng Chou et al. (+5 more)
Objective metrics for emotional expressiveness are vital for
speech generation, particularly in expressive synthesis and voice conversion requiring emotional prosody transfer. To quantify this, the field widely relies on
emotion similarity between reference and generated samples. This approach computes cosine similarity of
embeddings from encoders like
emotion2vec, assuming they capture affective cues despite linguistic and speaker variations. We challenge this assumption through controlled
adversarial tasks and human
alignment tests. Despite high classification accuracy, these latent spaces are unsuitable for zero-shot similarity evaluation. Representational limitations cause linguistic and speaker interference to overshadow emotional features, degrading discriminative ability. Consequently, the metric misaligns with human perception. This acoustic vulnerability reveals it rewards acoustic mimicry over genuine emotional synthesis.
Read the full abstract ▾
All authors Yun-Shao Tsai, Yi-Cheng Lin, Huang-Cheng Chou, Tzu-Wen Hsu, Yun-Man Hsu, Chun Wei Chen, Shrikanth Narayanan, Hung-yi Lee
Computation and Language
DSIPA: Detecting LLM-Generated Texts via Sentiment-Invariant Patterns Divergence Analysis
Siyuan Li, Aodu Wulianghai, Guangyan Li et al. (+5 more)
The rapid advancement of large
language models (LLMs) presents new security challenges, particularly in detecting machine-generated text used for misinformation, impersonation, and content forgery. Most existing detection approaches struggle with robustness against
adversarial perturbation, paraphrasing attacks, and domain shifts, often requiring restrictive access to model parameters or large labeled
datasets. To address this, we propose DSIPA, a novel training-free framework that detects LLM-generated content by quantifying sentiment distributional stability under controlled stylistic variation. It is based on the observation that LLMs typically exhibit more emotionally consistent outputs, while human-written texts display greater affective variation. Our framework operates in a zero-shot, black-box manner, leveraging two unsupervised metrics, sentiment distribution consistency and sentiment distribution preservation, to capture these intrinsic behavioral asymmetries without the need for parameter updates or probability access. Extensive experiments are conducted on state-of-the-art proprietary and open-source models, including GPT-5.2, Gemini-1.5-pro, Claude-3, and LLaMa-3.3. Evaluations on five domains, such as news articles, programming code, student essays, academic papers, and community comments, demonstrate that DSIPA improves F1 detection scores by up to 49.89% over baseline methods. The framework exhibits superior generalizability across domains and strong resilience to
adversarial conditions, providing a robust and interpretable behavioral signal for secure content identification in the evolving LLM landscape.
Read the full abstract ▾
All authors Siyuan Li, Aodu Wulianghai, Guangyan Li, Xi Lin, Qinghua Mao, Yuliang Chen, Jun Wu, Jianhua Li
Computation and Language · Cross-listed from cs.LG
Addressing Performance Saturation for LLM RL via Precise Entropy Curve Control
Bolian Li, Yifan Wang, Yi Ding et al. (+3 more)
Reinforcement learning (RL) has enabled complex
reasoning abilities in large
language models (LLMs). However, most RL
algorithms suffer from performance saturation, preventing continued gains as RL training scales. This problem can be characterized by the collapse of
entropy, a key diagnostic for exploration in RL. Existing attempts focus on preventing
entropy collapse through regularization or clipping. However, their resulting
entropy curves often exhibit instability in the long term, which hinders performance gains. In this paper, we introduce Entrocraft, a simple rejection-sampling approach that realizes user-customized
entropy schedule by biasing the advantage distributions. Entrocraft requires no objective regularization and is advantage-
estimator-agnostic. Theoretically, we relate per-step
entropy change to the advantage distribution under minimal assumptions. This explains the behavior of existing RL and
entropy-preserving methods. Entrocraft also enables a systematic study of
entropy schedules, which reveals that linear annealing, which starts high and decays to a slightly lower target, performs best. Empirically, Entrocraft addresses performance saturation, significantly improving generalization, output diversity, and long-term training. It enables a 4B model to outperform an 8B baseline, sustains improvement for up to 4x longer before plateauing, and raises pass@K by 50% over the baseline.
Read the full abstract ▾
All authors Bolian Li, Yifan Wang, Yi Ding, Anamika Lochab, Ananth Grama, Ruqi Zhang
Computation and Language
A Systematic Comparison of Prompting and Multi-Agent Methods for LLM-based Stance Detection
Genan Dai, Zini Chen, Yi Yang et al. (+1 more)
Stance detection identifies the attitude of a text author toward a given target. Recent studies have explored various LLM-based strategies for this task, from zero-shot prompting to multi-
agent debate. However, existing works differ in data splits, base models, and evaluation protocols, making fair comparison difficult. We conduct a systematic comparison that evaluates five methods across two categories –
prompt-based inference (Direct Prompting, Auto-CoT, StSQA) and
agent-based debate (COLA, MPRF) – on four
datasets with 14 subtasks, using 15 LLMs from six model families with parameter sizes from 7B to 72B+. Our experiments yield several findings. First, on all models with complete results, the best
prompt-based method outperforms the best
agent-based method, while
agent methods require 7 to 12 times more API calls per sample. Second, model scale has a larger impact on performance than method choice, with gains plateauing around 32B. Third,
reasoning-enhanced models (DeepSeek-R1) do not consistently outperform general models of the same size on this task.
Read the full abstract ▾
All authors Genan Dai, Zini Chen, Yi Yang, Bowen Zhang
Computation and Language
Classification of Public Opinion on the Free Nutritional Meal Program on YouTube Media Using the LSTM Method
Berliana Enda Putri, Lisa Diani Amelia, Muhammad Zaky Zaiddan et al. (+3 more)
Public opinion towards the Free Nutritious Meal Program (MBG) on YouTube social media reflects diverse community responses. This study applies the Long Short-Term
Memory (LSTM) method to classify sentiments from 7,733 YouTube comments. The results show that the LSTM model achieves 89% accuracy, with strong performance on negative sentiment (F1-score 0.94) but weaker performance on positive sentiment (F1-score 0.55) due to class imbalance, as negative data account for 87.7% of the
dataset. These findings confirm the effectiveness of LSTM for sentiment analysis of Indonesian text while highlighting the challenge of imbalanced data. This research contributes to social media-based public policy evaluation
Read the full abstract ▾
All authors Berliana Enda Putri, Lisa Diani Amelia, Muhammad Zaky Zaiddan, Luluk Muthoharoh, Ardika Satria, Martin Clinton Tosima Manullang
Computation and Language
Benchmarking PyCaret AutoML Against BiLSTM for Fine-Grained Emotion Classification: A Comparative Study on 20-Class Emotion Detection
Arya Muda Siregar, Arielva Simon Siahaan, Haikal Fransisko Simbolon et al. (+3 more)
Fine-grained
emotion classification, which identifies specific emotional states such as happiness, anger, sadness, and fear, remains a challenging task in natural
language processing. This study
benchmarks classical machine learning and deep learning approaches for 20-class
emotion classification using the 20-
Emotion Text Classification
Dataset containing 79,595 English sentences. On the machine learning side, Logistic Regression, Multinomial Naive Bayes, and Support Vector Machine are evaluated using TF-IDF features. On the deep learning side, Bidirectional Long Short-Term
Memory, Gated Recurrent Unit, and a lightweight
Transformer implemented in PyTorch are compared. The results show that BiLSTM achieves the best overall performance with 89% accuracy and a weighted F1-score of 0.89, slightly outperforming the best machine learning model, SVM, which reaches 88.11% accuracy. The findings indicate that while traditional machine learning models remain competitive and computationally efficient, sequence-based deep learning models better capture contextual emotional cues in text.
Read the full abstract ▾
All authors Arya Muda Siregar, Arielva Simon Siahaan, Haikal Fransisko Simbolon, Luluk Muthoharoh, Ardika Satria, Martin C.T. Manullang
Computation and Language
Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference
Vasu Shyam, Anna Golubeva and Quentin Anthony
We present tensor and sequence parallelism (TSP), a parallel execution strategy that folds tensor parallelism and sequence parallelism onto a single device axis. In conventional multi-dimensional parallelism layouts, tensor parallelism (TP) shards model weights while sequence parallelism (SP) shards
tokens, reducing per-device parameter or activation
memory, respectively. Traditionally, each scheme is assigned its own mesh dimension. TSP instead assigns each rank both a weight shard and a sequence shard, reducing both parameter and activation
memory along the same device axis. We implement this design with two runtime schedules. For
attention, ranks iterate over broadcast parameter shards and reconstruct context through a sequence-wise key/value exchange. For gated MLPs, weight shards circulate in a ring while partial outputs accumulate locally. By sharding both weights and activations across the same devices, TSP trades additional communication volume for reduced
memory overhead. We provide a theoretical communication and
memory analysis, describe our implementation of TSP
attention and gated MLP blocks, and
benchmark TSP against TP, SP, and TP+SP. These results position TSP as a hardware-aware alternative for long-context and
memory-constrained model training, and as a viable axis of parallelism in concert with existing parallelism schemes such as pipeline and expert parallelism for dense and mixture-of-expert models.
Read the full abstract ▾
Computation and Language
Calibrated Surprise: An Information-Theoretic Account of Creative Quality
Bo Zou and Chao Xu
In the era of large
language models, creative writing quality lacks a computable theoretical anchor. The dominant approaches are rubric scoring – decomposing holistic aesthetic judgment into sub-scores – and RLHF preference signals – replacing quality with group votes. Both bypass the statistical structure of the text itself. This paper provides an information-theoretic foundation to fill this gap. We propose 'calibrated surprise' as the information-theoretic essence of excellent creative writing. This judgment matches reading intuition and covers its opposite. This literary judgment admits a precise mathematical formulation. Under full-dimensional constraints Y, feasible writing choices are forced into an extremely narrow space. The rare survivors are, from the unconstrained perspective, exactly the least predictable choices. Both are measured precisely by Shannon mutual information I(X;Y) = H(X) - H(X|Y) – 'calibrated' corresponds to H(X|Y) approaching 0; 'surprising' corresponds to H(X) going high. The subtraction structure of the formula naturally separates 'well-grounded surprise' from 'pure noise'. We use
token-level logprobs from Qwen1.5-7B as an operational proxy for the ideal reader's probability distribution. Across 20 pairs (12 Chinese / 8 English) of high-quality vs. systematically degraded literary passages, 20/20 pairs support the core prediction: high-quality passages have systematically higher I(X;Y) than their degraded versions.
Read the full abstract ▾
Computation and Language
FlowBot: Inducing LLM Workflows with Bilevel Optimization and Textual Gradients
Hongyeon Yu, Young-Bum Kim and Yoon Kim
LLM workflows, which coordinate structured calls to individual LLMs/
agents to achieve a particular goal, offer a promising path towards building powerful AI systems that can tackle diverse tasks. However, existing approaches for building such workflows generally rely on human-crafted pipelines and
prompts, which presents a substantial bottleneck in real world deployment. How can we automatically induce LLM-based
agents and workflows in a data-driven way? This paper describes a simple data-driven approach for automatically inducing
agents and LLM workflows. We formulate workflow induction as a bilevel optimization problem: an outer loop which optimizes a high-level sketch of the workflow (in particular how the LLM calls should be structured), and an inner loop which optimizes each individual LLM call one-by one. Both loops are optimized with “textual gradients” where for the inner loop we optimize each component in a modular way through “backpropagating” textual gradients layer-by-layer. We find that LLM workflows discovered through our FlowBot (workflow induction through bilevel optimization and textual gradients) approach performs competitively against strong baselines that make use of human-crafted or generated workflows.
Read the full abstract ▾
Computation and Language
StratMem-Bench: Evaluating Strategic Memory Use in Virtual Character Conversation Beyond Factual Recall
Yerong Wu, Tianxing Wu, Minghao Zhu et al. (+2 more)
Achieving realistic human-like conversation for virtual characters requires not only a simple memorization and recall of past events, but also the strategic utilization of
memory to meet factual needs and social engagement. Current
memory utilization relevant (e.g.,
memory-augmented generation, long-term dialogue, and etc.)
benchmarks overlook this nuance, treating
memory primarily as a static repository of facts rather than a dynamic resource to be strategically deployed in dialogues. To address this gap, we design StratMem-Bench, a new
benchmark to evaluate strategic
memory use in character-centric dialogues. This
dataset comprises 657 instances where virtual characters must navigate heterogeneous
memory pools containing required, supportive, and irrelevant
memories. We also propose a framework with different evaluation metrics including Strict
Memory Compliance,
Memory Integration Quality, Proactive Enrichment Score and Conditional Irrelevance Rate, to evaluate strategic
memory use capabilities of virtual characters. Experiments on StratMem-Bench which leverage the state-of-the-art large
language models as virtual characters show that all models perform well at distinguishing between required and irrelevant
memories, but struggle once supportive
memories are introduced into the decision process.
Read the full abstract ▾
All authors Yerong Wu, Tianxing Wu, Minghao Zhu, Hangyu Sha, Haofen Wang
Computation and Language · Cross-listed from cs.CR
LATTICE: Evaluating Decision Support Utility of Crypto Agents
Aaron Chan, Tengfei Li, Tianyi Xiao et al. (+3 more)
We introduce
LATTICE, a
benchmark for evaluating the decision support utility of crypto
agents in realistic user-facing scenarios. Prior crypto
agent benchmarks mainly focus on
reasoning-based or outcome-based evaluation, but do not assess
agents' ability to assist user decision-making.
LATTICE addresses this gap by: (1) defining six evaluation dimensions that capture key decision support properties; (2) proposing 16 task types that span the end-to-end crypto copilot workflow; and (3) using LLM judges to automatically score
agent outputs based on these dimensions and tasks. Crucially, the dimensions and tasks are designed to be evaluable at scale using LLM judges, without relying on ground truth from expert annotators or external data sources. In lieu of these dependencies,
LATTICE's LLM judge rubrics can be continually audited and updated given new dimensions, tasks, criteria, and human feedback, thus promoting reliable and extensible evaluation. While other
benchmarks often compare foundation models sharing a generic
agent framework, we use
LATTICE to assess production-level
agents used in actual crypto copilot products, reflecting the importance of orchestration and UI/UX design in determining
agent quality. In this paper, we evaluate six real-world crypto copilots on 1,200 diverse queries and report breakdowns across dimensions, tasks, and query categories. Our experiments show that most of the tested copilots achieve comparable aggregate scores, but differ more significantly on dimension-level and task-level performance. This pattern suggests meaningful trade-offs in decision support quality: users with different priorities may be better served by different copilots than the aggregate rankings alone would indicate. To support reproducible research, we open-source all
LATTICE code and data used in this paper.
Read the full abstract ▾
All authors Aaron Chan, Tengfei Li, Tianyi Xiao, Angela Chen, Junyi Du, Xiang Ren