Computation and Language · Cross-listed from cs.HC
Reflecti-Mate: A Conversational Agent for Adaptive Decision-Making Support Through System 1 and System 2 Thinking
Morita Tarvirdians, Senthil Chandrasegaran, Hayley Hung et al. (+2 more)
Making high-stakes personal decisions involves cognitive, emotional, and intuitive processes, and individuals differ in how they allocate
attention across these modes. Integration of these processes has shown to benefit decision making. Yet, most current decision-support systems focus primarily on supporting cognitive aspects, rather than adapting to the individual's thinking profile to support integration of different types of thoughts. In this study, we investigate an
agent designed to encourage integration by adapting to the individual user's thought patterns. We explore its effects on participants' perceptions of the
agent and their reflective behavior, in comparison with unaided pre-reflection and a baseline
agent. In a between-subjects study (N = 128), our
agent, which fostered broad and elaborated thinking, enabled more personalized reflective trajectories, elicited more integrative reflective
language, and was perceived as providing stronger support for holistic reflection. In contrast, the baseline
agent produced homogenized profiles dominated by cognitive
language across participants.
Read the full abstract ▾
All authors Morita Tarvirdians, Senthil Chandrasegaran, Hayley Hung, Catholijn M. Jonker, Catharine Oertel
Computation and Language
BeLink: Biomedical Entity Linking Meets Generative Re-Ranking
Darya Shlyk, Stefano Montanelli and Lawrence Hunter
Despite recent progress, Biomedical Entity Linking (BEL) with large
language models (LLMs) remains computationally inefficient and challenging to deploy in practical settings. In this work, we demonstrate that instruction-tuning of open-source generative models can offer an effective solution when applied at the re-ranking stage of the BEL pipeline. We propose a set-wise instruction-tuning formulation that enables fast and accurate candidate selection. Our method demonstrates strong performance on multiple BEL
benchmarks, yielding significant improvements in linking accuracy (3%-24%) while reducing inference time compared to the state-of-the-art. We integrate our generative re-ranker into BeLink, a modular, end-to-end system designed for practical real-world BEL applications.
Read the full abstract ▾
Computation and Language
Polite on the Surface, Broken in Practice: A Curated Dataset for Fixing Generation and Register Failures in Low-Resource Bangla Text Generation
Md. Asaduzzaman Shuvo, Mahedi Hasan, Md. Tashin Parvez et al. (+2 more)
Recent advances in
Multilingual Large
Language Models (MLLMs) have significantly enhanced cross-lingual conversational capabilities, yet modeling culturally nuanced and context-dependent communication remains a critical bottleneck. Specifically, existing state-of-the-art models exhibit a severe pragmatic gap when handling structural variations, regional idioms, and honorific consistencies in low-resource contexts like Bangla. To address this limitation, we introduce a novel, culturally aligned instruction-tuning
dataset for BangLa Application and DialoguE generation - BLADE and benchmarking framework comprising $4,196$ meticulously curated interaction pairs. We leverage this resource to systematically fine-tune and evaluate leading open-weight architectures, including DeepSeek-8B and LLaMA-3.2-3B, utilizing parameter-efficient fine-tuning via LoRA adapters in a 4-bit NormalFloat (NF4) quantization framework. Our empirical evaluations demonstrate that models fine-tuned on our
dataset yield substantial improvements in structural fidelity and honorific
alignment, providing a rigorous
benchmark for bridging pragmatic disparities in low-resource
multilingual text generation. Code and
dataset: https://github.com/ashuvo25/Bangla_Application_LLM/tree/main
Read the full abstract ▾
All authors Md. Asaduzzaman Shuvo, Mahedi Hasan, Md. Tashin Parvez, Azizul Haque Noman, Md. Shafayet Hossain Ovi
Computation and Language · Cross-listed from cs.LG
Structured-Sparse Attention for Entity Tracking with Subquadratic Sequence Complexity
Hangyue Zhao, Paul Caillon, Erwan Fagnou et al. (+1 more)
Entity tracking requires maintaining and updating latent states for entities and attributes over long sequences. Recent task-specific
attention operators can compress deep
Transformer stacks into a few layers by performing multi-hop state propagation within a single layer, but their dense evaluation remains expensive. We show that in this setting, learned
attention is strongly structured: most mass concentrates in local block-diagonal neighborhoods with a light cross-block residue. Exploiting this, we derive a blockwise evaluation of a resolvent-style operator that keeps within-block interactions exact and routes cross-block interactions through a reduced system. The resulting evaluation is subquadratic in sequence length $O(n^{4/3}d)$ (and $O(n^{7/3})$ when $d\approx n$). On controlled tracking
benchmarks, our method matches the dense operator's accuracy while reducing wall-clock time by $12-29\%$ under a standardized measurement protocol, and is up to $2.4 \times$ faster than a compact dense
Transformer at comparable exact-match accuracy. We further provide ablations over block size and model capacity, and identify a limitation: performance collapses when the number of simultaneously evolving properties exceeds the number of
attention heads.
Read the full abstract ▾
All authors Hangyue Zhao, Paul Caillon, Erwan Fagnou, Alexandre Allauzen
Computation and Language
In Silico Modeling of the RAMPHO Buffer: Dissociating Informational and Energetic Masking via Phonetic Entropy in Deep Neural Networks
Stefan Bleeck
The fundamental challenge of listening in multi-talker environments is a cognitive bottleneck, defined by the Ease of
Language Understanding (ELU) model as a failure within the RAMPHO episodic buffer. Current deep neural networks for
speech enhancement optimize purely for physical acoustics, failing to account for the cognitive penalty of informational masking. Here, we present an in silico simulation of the RAMPHO buffer using the frame-by-frame phonetic
entropy of a self-supervised acoustic model (wav2vec 2.0). By contrasting a semantically intact distractor with a phase-decorrelated distractor (the Concentration Shield) across a signal-to-noise ratio (SNR) sweep, we successfully dissociate the cognitive penalty of informational distraction from the physical penalty of energetic decay. The simulation reveals a cognitive-acoustic Pareto optimization problem: destroying a distractor's semantic payload provides a release from informational masking at high SNRs, but fundamentally degrades temporal glimpsing cues at low SNRs.
Read the full abstract ▾
Computation and Language
From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models
Caleb Munigety
We propose a five-stage methodology for causal feature analysis in
transformer language models (probe design, feature extraction, causal validation, robustness testing, and deployment integration) and demonstrate it end-to-end on GPT-2 small performing the Indirect Object Identification (IOI) task. Activation patching recovers the canonical IOI
circuit (layer-9 head 9 alone gives recovery +1.02). A sparse autoencoder recovers per-name selective features with effect sizes of 30 to 50 activation units. Causal validation finds these features specifically but only partially causal: ablating fifteen of them leaves the model accurate on 98% of
prompts. Two NLA-inspired evaluations strengthen this picture: the fifteen selective features explain only 31% of activation variance versus the SAE's 99.7%, and selectivity ratio anticorrelates with causal force (r = -0.56). Robustness testing under three distribution shifts finds that the
circuit transfers cleanly but feature ablation effects degrade substantially, exposing a gap between detection robustness and causal robustness. A cost-based deployment evaluation (assumed $50/FN, $0.42/FP, 2% error rate) finds an optimal monitor configuration yielding $8.96 per 1000 queries against a $1000 baseline, a 99.1% saving. Optimal composition strategy varies with cost ratio and base rate. The conjunction of stages produces findings no single stage would.
Read the full abstract ▾
Computation and Language
Cohesion-6K: An Arabic Dataset for Analyzing Social Cohesion and Conflict in Online Discourse
Aisha Ali Al-Athba and Wajdi Zaghouani
The study of online discourse has become central to understanding societal polarization. While much research has focused on detecting overt toxicity, the subtle dynamics of social cohesion, meaning the interaction between divisive and unifying narratives, remain computationally underexplored (Bail, 2021; Gonzalez-Bailon and Lelkes, 2023). This paper presents Cohesion-6K, a manually and ChatGPT-assisted annotated
dataset of six thousand Arabic public Facebook posts related to the Israeli Occupation of Palestine. Each post is assigned to one of five discourse categories that represent a continuum from conflict to cohesion: Conflict, Resolution, Community Engagement, Supportive Interactions, and Shared Values. The annotation process combines expert human judgment with model-assisted pre-labeling verified by trained annotators, achieving substantial inter-annotator agreement (Cohens kappa = 0.85). Quantitative analysis reveals a consistent engagement gap, where conflict-oriented posts receive between two and four times more user interaction than resolution-oriented ones (p < 0.01). This pattern illustrates how divisive discourse tends to attract disproportionate visibility in Arabic social media spaces. Cohesion-6K provides a transparent and reproducible resource for the study of online cohesion and polarization. The
dataset, annotation guidelines, and preprocessing code will be released for research use under an open license, supporting future work in computational social science, digital communication, and Arabic natural
language processing.
Read the full abstract ▾
Computation and Language
Assisted Counterspeech Writing at the Crossroads of Hate Speech and Misinformation
Genoveffa Martone, Helena Bonaldi and Marco Guerini
Hate
speech and misinformation frequently co-occur online, amplifying prejudice and polarization. Given their scale, using Large
Language Models (LLMs) to assist expert counterspeech (CS) writing has gained interest, yet prior work has addressed these phenomena separately. We bridge this gap by studying CS generation in contexts where both hate and misinformation co-occur. We test three knowledge-driven generation strategies: first we
prompt an LLM with fact-checkers' guidelines and fact-checking articles; secondly, with NGOs' guidelines and reports; thirdly, we create a mixed strategy that combines guidelines and documents from both. 23 experts revise the generated CS, which are assessed via human and automatic metrics. While LLMs produce adequate CS in 40% of cases, expert edits substantially improve naturalness, exhaustiveness, and adherence to guidelines. Based on the post-edited CS, the mixed strategy proves to be the most effective in crowdsourcing evaluation, pairing strong factual correction with stereotype mitigation and empathetic engagement. We release a
dataset of hateful and misinformed claims with expert-verified CS and supporting knowledge.
Read the full abstract ▾
Computation and Language
DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Agent Memory
Jianing Yin, Tan Tang and Yingcai Wu
Large
language model (LLM)
agents still struggle to effectively use long-term
memory, with answer-supporting evidence often scattered across long conversational histories and buried in substantial irrelevant content. Existing
memory systems commonly process
memory into dedicated units before future queries are known and retrieve these preconstructed units at query time. Because the contents and granularity of these units are determined before the current query is known, the retrieved units can remain coarse and noisy for that query, leaving downstream answerers to further denoise them and uncover query-specific evidence. We present DeferMem, a long-term
memory framework that decouples this problem into high-recall candidate
retrieval and query-conditioned evidence
distillation. DeferMem uses a lightweight segment-link structure to organize raw history and retrieve broad candidates at query time. A
memory distiller then distills these high-recall but highly noisy candidates into a set of faithful, self-contained, and query-conditioned evidence. To train this distiller, we introduce DistillPO, a reinforcement learning
algorithm that formulates post-
retrieval evidence
distillation as a structured action comprising message selection and evidence rewriting. It optimizes this action with a decomposed-and-gated reward pipeline and structure-aligned advantage assignment, gating reward components from validity to quality checks while exposing answerability-related feedback early and assigning each reward to its responsible output span. On LoCoMo and LongMemEval-S, DeferMem surpasses strong baselines in QA accuracy and
memory-system efficiency, achieving the highest QA accuracy and fastest runtime while consuming no commercial-API
tokens for
memory operations.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Epicure: Navigating the Emergent Geometry of Food Ingredient Embeddings
Jakub Radzikowski and Josef Chen
We present Epicure, a family of three sibling skip-gram ingredient
embeddings retrained from scratch on a
multilingual recipe corpus. We aggregate 4.14M recipes from 11 sources spanning seven
languages, English, Chinese, Russian, Vietnamese, Spanish, Turkish, Indonesian, German, and Indian-English, and normalise the raw ingredient strings to 1,790 canonical entries via an LLM-augmented pipeline. A 203,508-edge ingredient-ingredient NPMI graph and an 80,019-edge typed FlavorDB ingredient-compound graph, 2,247 typed compound nodes across 15 categories, seed three Metapath2Vec variants that share architecture and hyperparameters and differ only in the random-walk schema: Cooc walks the co-occurrence graph only, Chem walks the typed compound metapaths only, and Core blends both via injected ingredient-ingredient walks at controlled mixing, placing each model at a distinct point on the chemistry-vs-recipe-context spectrum.
Read the full abstract ▾
Computation and Language
Unified Data Selection for LLM Reasoning
Xiaoyuan Li, Yubo Ma, Chengpeng Li et al. (+6 more)
Effectively training Large
Language Models (LLMs) for complex, long-CoT
reasoning is often bottlenecked by the need for massive high-quality
reasoning data. Existing methods are either computationally expensive or fail to reliably distinguish high- from low-quality
reasoning samples. To address this, we propose High-
Entropy Sum (HES), a training-free metric that quantifies
reasoning quality by summing only the
entropy of the top (e.g., 0.5%) highest-
entropy tokens in each
reasoning sample. We validate HES across three mainstream training paradigms: Supervised Fine-tuning (SFT), Rejection Fine-tuning (RFT), and Reinforcement Learning (RL), with extensive results demonstrating its consistent effectiveness and significantly reduced computational overhead. In SFT, training on the top 20% HES-ranked data matches full-
dataset performance, while using the lowest-HES data degrades it. In RFT, our HES-based training approach significantly outperforms baseline methods. In RL, HES-selected successful trajectories enable the model to learn strong
reasoning patterns, significantly surpassing other compared methods. Our findings establish HES as a robust, training-free metric that enables a unified, effective, and efficient method for developing advanced
reasoning in LLMs.
Read the full abstract ▾
All authors Xiaoyuan Li, Yubo Ma, Chengpeng Li, Fengbin Zhu, Yiyao Yu, Keqin Bao, Wenjie Wang, Fuli Feng, Dayiheng Liu
Computation and Language
Multi-Stage Training for Abusive Comment Detection in Indic Languages
Pranshu Rastogi, Madhav Mathur, Ramaneswaran S et al. (+1 more)
In recent years social media has become an increasingly popular tool for communication. People use it to share their ideas, exchange information, and discuss thoughts. Given its prevalence and widespread reach, social media must remain a safe space for people. Content generated on social media can be abusive and it has become increasingly important to detect such content. In this paper, we use a
language-based preprocessing and an ensemble of several models and analyze their performance of abusive comment detection. Through extensive experimentation, we propose a pipeline that minimizes the false-positive rate (marking non-abusive as abusive) so that these systems can detect abusive comments without undermining the freedom of expression.
Read the full abstract ▾
All authors Pranshu Rastogi, Madhav Mathur, Ramaneswaran S, Kshitij Mohan
Computation and Language · Cross-listed from cs.LG
Boundary-targeted Membership Inference Attacks on Safety Classifiers
Anthony Hughes, Alexander Goldberg, Prince Jha et al. (+3 more)
Safety classifiers are essential safeguards within generative AI systems, filtering harmful content or identifying at-risk users when interacting with large
language models. Despite their necessity, these models are trained on sensitive
datasets including discussions of self-harm and mental health, raising important, yet poorly understood,
privacy concerns. Membership inference attacks (MIAs) allow adversaries to infer membership of examples used to train models. In this work, we hypothesize that identifying the examples on which the classifier is least confident are informative for an adversary to infer membership. This reflects a localized failure of generalization, where the model relies on memorization to resolve ambiguity in the training set. To investigate this, we introduce a new boundary-targeted selection strategy that identifies low confidence examples that amplify the signal of an examples membership within a training set. Our experimental results show that an adversary can recover 19% of the conversations a safety classifier flagged as indicating user distress, at a 5% false-positive rate, on a classifier fine-tuned for detecting a user who may require emotional support. This is $3.5$ times more than attacking using state-of-the-art MIA methods alone. Finally, we characterize the boundary laying examples and show that content-based filtering is ineffective for protection, and existing noise strategies can effectively mitigate susceptibility of these examples.
Read the full abstract ▾
All authors Anthony Hughes, Alexander Goldberg, Prince Jha, Adam Perer, Nikolaos Aletras, Niloofar Mireshghallah
Computation and Language
Modeling Pathology-Like Behavioral Patterns in Language Models Through Behavioral Fine-Tuning
Nicola Milano and Davide Marocco
Large
language models are increasingly used as computational tools for modeling human-like behavior. We introduce a behavioral induction framework that modifies model policies through fine-tuning on structured decision-making tasks: using synthetic
datasets inspired by maladaptive behavioral patterns, including depression and paranoia, we train
transformer-based
language models to consistently select specific classes of actions across diverse contexts. We then test whether this behavioral optimization produces systematic changes in generative distributions. Across two architectures, fine-tuned models show stable, context-general shifts in next-
token probability distributions, including increased probability assigned to negative and threat-related interpretations in open-ended
language tasks. These effects generalize beyond training contexts and are detectable in qualitative completions, psychometric-style evaluations, and quantitative distributional metrics such as Jensen-Shannon divergence. Induced behavioral profiles also show partial specificity. Models optimized for different behavioral patterns exhibit dissociable response tendencies across evaluation probes, suggesting that structured behavioral training produces differentiated policy-level biases rather than generic distributional skew. We interpret these findings as evidence that consistent behavioral optimization in LLMs can generate stable behavioral and distributional patterns consistent with altered latent priors, linking action selection and
language generation. More broadly, the results support a view of LLMs as policy-based systems in which behavioral constraints shape emergent representational structure, highlighting their potential as controlled testbeds for studying the relationship between behavior, interpretation, and generative
language in computational models of cognition.
Read the full abstract ▾
Computation and Language
TransitLM: A Large-Scale Dataset and Benchmark for Map-Free Transit Route Generation
Hanyu Guo, Jiedong Yang, Chao Chen et al. (+3 more)
Public transit route planning traditionally depends on structured map infrastructure and complex routing engines, and no existing
dataset supports training models to bypass this dependency. We present TransitLM, a large-scale
dataset of over 13 million transit route planning records from four Chinese cities covering 120,845 stations and 13,666 lines, released as a continual pre-training corpus and
benchmark data for three evaluation tasks with complementary metrics. Experiments show that an LLM trained on TransitLM produces structurally valid routes at high accuracy and implicitly grounds arbitrary GPS coordinates to appropriate stations without any explicit mapping. These results demonstrate that transit route planning can be learned entirely from data, enabling end-to-end, map-free route generation directly from origin-destination information. The
dataset and
benchmark are available at https://huggingface.co/
datasets/GD-ML/TransitLM, with evaluation code at https://github.com/HotTricker/TransitLM.
Read the full abstract ▾
All authors Hanyu Guo, Jiedong Yang, Chao Chen, Longfei Xu, Kaikui Liu, Xiangxiang Chu
Computation and Language
Pattern-and-root inflectional morphology: the Arabic broken plural
Alexis Amid Neme and Eric Laporte
We present a substantially implemented model of description of the inflectional morphology of Arabic nouns, with special
attention to the management of dictionaries and other
language resources by Arabic-speaking linguists. The breakthrough lies in the reversal of the traditional root-and-pattern Semitic model into pattern-and-root, giving precedence to patterns over roots. Our model includes broken plurals (BPs), i.e. plurals formed by modifying the stem. It is based on the traditional notions of root and pattern of Semitic morphology. However, as compared to traditional Arabic morphology, it keeps the formal description of inflection separate from that of derivation and semantics. As traditional Arabic dictionaries, the updatable dictionary is structured in lexical entries for lemmas, and the reference spelling is fully diacritized. In our model, morphological analysis of Arabic text is performed directly with a dictionary of words and without morphophonological rules. Our taxonomy for noun inflection is simple, orderly and detailed. We simplify the taxonomy of singular patterns by specifying vowel quantity as v or vv, and ignoring vowel quality. Root alternations and orthographical variations are encoded independently from patterns and in a factual way, without deep roots or morphophonological or orthographical rules. Nouns with a triliteral BP are classified according to 22 patterns subdivided into 90 classes, and nouns with a quadriliteral BP according to 3 patterns subdivided into 70 classes. These 160 classes become 300 inflectional classes when we take into account inflectional variations that affect only the singular. We provide a straightforward encoding scheme that we applied to 3 200 entries of BP nouns.
Read the full abstract ▾
Computation and Language
Harder to Defend: Towards Chinese Toxicity Attacks via Implicit Enhancement and Obfuscation Rewriting
Jingyi Kang, Junyu Lu, Bo Xu et al. (+4 more)
Large
language models (LLMs) require robust toxicity evaluation beyond explicit wording. This setting remains underexplored in Chinese, where toxicity may combine semantic indirectness with surface obfuscation. We introduce Chinese Implicit Toxicity Attack (CITA), a controlled red-team evaluation and defense-data generation framework, not a deployable evasion tool. CITA uses three stages: (i) Harmful Intent Learning, (ii) Implicit Toxicity Enhancement, and (iii) Obfuscation Variant Rewriting, to preserve harmful intent, increase implicitness, and add controlled surface variants. On CITA-generated evaluation samples, the seven tested detectors exhibit substantial missed-detection risks, reaching an average ASR of 69.48%; human evaluation further confirms preserved harmfulness and increased implicitness/evasiveness. As a downstream defense application, we fine-tune a Chinese Implicit Toxicity Defense model (CITD) with CITA-generated red-team data, showing that such data can improve robustness through additional training.
Read the full abstract ▾
All authors Jingyi Kang, Junyu Lu, Bo Xu, Hongbo Wang, Linlin zong, Roy Ka-Wei Lee, Hongfei Lin
Computation and Language
IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions
Kai Golan Hashiloni, Daniel Fadlon, Lior Livyatan et al. (+3 more)
Idioms pose a fundamental challenge for
language models, as their meaning cannot be inferred from surface form alone. Understanding such expressions, therefore, requires semantic abstraction beyond lexical overlap. We introduce IdioLink, a
retrieval benchmark designed to test whether models can link idiomatic expressions to conceptually equivalent meanings expressed in literal or paraphrased forms. IdioLink comprises 10,700 documents and 2,140 queries, spanning 107 idioms with both literal and figurative uses. Each document and query is annotated with spans that convey the core meaning. Evaluating strong
embedding baselines (e.g., BGE, E5, Contriever, and Qwen), we show that current models struggle to retrieve equivalent meanings across divergent surface realizations, relying instead on topical and shallow semantic cues. IdioLink exposes key gaps in idiom-aware semantic
retrieval and provides a challenging testbed for future models.
Read the full abstract ▾
All authors Kai Golan Hashiloni, Daniel Fadlon, Lior Livyatan, Ofri Hefetz, Jiahuan Pei, Kfir Bar
Computation and Language
GHI: Graphormer over Conditioned Hypergraph Incidence for Aspect-Based Sentiment Analysis
Yu Du, Wenlong Zhu, Xingze Li et al. (+3 more)
Aspect-based sentiment analysis (ABSA) requires models to bind sentiment evidence to the correct aspect, making it a natural testbed for fine-grained structural
reasoning. We introduce GHI, a Graphormer-over-Conditioned-Hypergraph-Incidence framework that is designed as an incidence-based structural
reasoning layer built on a bipartite
topology. GHI represents diverse linguistic and semantic evidence as
token–hyperedge incidence relations, allowing different structural signals to be incorporated through a unified interface. Extensive experiments on six standard ABSA
benchmarks show that GHI outperforms all baselines on the SemEval domains, and multi-seed evaluations show stable improvements over strong DeBERTa. Further experiments show that with only 247M parameters, GHI approaches the performance of 11B Flan-T5 based methods on the ISE
benchmark. Moreover, it demonstrates strong robustness on the challenging ARTS
datasets, maintaining highly competitive performance where traditional models degrade. These results demonstrate that compact structural
reasoning remains a valuable alternative to scale-driven approaches for fine-grained tasks.
Read the full abstract ▾
All authors Yu Du, Wenlong Zhu, Xingze Li, Chenglong Cao, Jing Wang, Yukun Ma
Computation and Language · Cross-listed from cs.LG
Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RL
Sophia Xiao Pu, Zhaotian Weng, Chengzhi Liu et al. (+4 more)
Self-play reinforcement learning trains
language models on their own generated tasks, co-evolving a proposer and solver without human labels. Recent systems report strong
reasoning gains, but collapse and instability are widely observed and poorly understood. The dominant response treats this as a reward-design problem. We argue instead that self-play stability is governed by two distinct levers: a data-level gate that decides which proposer-generated tasks enter the training pool, and the reward signal that updates the policy on tasks already admitted. Through controlled experiments on a Python output-prediction task and a deterministic-DSL twin task that strips pretraining priors, output ambiguity, and executor noise, we find the two levers are asymmetric. A strict gate is sufficient for stability under every reward variant we test, including a self-consistency reward with no access to ground truth; while no reward variant is sufficient once the gate is removed. This asymmetry exposes a counter-intuitive coupling we call the Grounded Proposer Paradox: a proposer with ground-truth access accelerates collapse faster than an ungrounded one when paired with a self-consistency solver, by concentrating training on clean tasks that form the fastest path to a spurious self-consistent attractor. Replacing the binary gate with a continuous strictness parameter $\varepsilon$ further reveals a two-stage phase transition: training-side metrics decouple at low $\varepsilon$, while validation accuracy holds until $\varepsilon$ is much higher. Data-level gating, not reward calibration, is the binding constraint on self-play stability.
Read the full abstract ▾
All authors Sophia Xiao Pu, Zhaotian Weng, Chengzhi Liu, Jayanth Srinivasa, Gaowen Liu, William Yang Wang, Xin Eric Wang
Computation and Language
Audience Engagement with Arabic Women's Social Empowerment and Wellbeing: A Decadal Corpus
Wajdi Zaghouani, Mabrouka Bessghaier, MD. Rafiul Biswas et al. (+1 more)
This paper presents the Arabic Women and Society Corpus, a ten year collection of 252,487 public Arabic Facebook posts related to women's empowerment and social wellbeing. The corpus was collected from 51,660 pages across 77 countries between 2013 and 2024, resulting in more than 267 million user interactions. Each post includes engagement metrics such as shares, comments, and emotional reactions, providing a unique view of audience sentiment and social
attention. The data were processed using an automated pipeline with
language identification, normalization, and metadata cleaning to ensure reliability and reproducibility. The corpus enables large scale analysis of gender discourse, social reform, and emotional engagement across Arabic dialects. It supports research in Arabic natural
language processing, computational social science, and digital communication studies. The
dataset and accompanying documentation will be released under request for research use.
Read the full abstract ▾
All authors Wajdi Zaghouani, Mabrouka Bessghaier, MD. Rafiul Biswas, Shimaa Amer Ibrahim
Computation and Language
Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents
Sovandara Chhoun, Pichdara Po, Sereiwathna Ros et al. (+2 more)
In this study, we compare the performance of four text chunking approaches: Recursive, Khmer-Aware, Sentence-Based, and LLM-Based within a
Retrieval-Augmented Generation (RAG) framework applied to Khmer agricultural documents. The document chunks are encoded using the BGE-M3
multilingual embedding model and retrieved using the FAISS library. Performance is evaluated using four metrics: Average
Retrieval Score (L2 distance), Answer Relevance, Khmer Coverage, and Khmer Intersection over Union, all measured against ground-truth question-answer pairs. For evaluation, we perform 5-fold cross-validation over 18 question-answer pairs. We observe the best performance for the character-based Recursive chunking method with a chunk size of 300 characters, achieving the lowest L2 distance (0.4295 +- 0.0461), highest Answer Relevance (0.8663 +- 0.0199), and highest Khmer IoU (0.6441 +- 0.0347). A paired t-test shows a statistically significant improvement over the Sentence-Based chunking method in L2 distance (p = 0.0121). These results highlight the importance of
segmentation granularity and structural preservation for optimizing dense
retrieval in morphologically complex, low-resource
languages such as Khmer.
Read the full abstract ▾
All authors Sovandara Chhoun, Pichdara Po, Sereiwathna Ros, Wan-Sup Cho, Saksonita Khoeurn
Computation and Language
Structure Retention in Embedding Spaces as a Predictor of Benchmark Performance
Amanda Myntti, Jenna Kanerva, Veronika Laippala et al. (+1 more)
In this paper, we show that high-performing
embedding models organize their
embedding spaces in a consistent way. We evaluate 25 contemporary
embedding models on five MTEB tasks spanning four diverse task categories (
retrieval, bitext mining, pair classification, and summarization) in both English and
multilingual settings, and reveal that nearest-neighbor overlap and magnitude differences in independent component analysis (ICA) between paired text instances strongly correlate (even up to 0.97) with performance on the given task. Ultimately, we show that
embedding tasks display varying degrees of linearity and reliance on retention of local information. Our results further the understanding of
embeddings, their relation to model performance, and shed light on possible future training objectives and optimizing conditional
embeddings.
Read the full abstract ▾
All authors Amanda Myntti, Jenna Kanerva, Veronika Laippala, Filip Ginter
Computation and Language · Cross-listed from cs.LG
Maestro: Reinforcement Learning to Orchestrate Hierarchical Model-Skill Ensembles
Jinyang Wu, Guocheng Zhai, Ruihan Jin et al. (+7 more)
The proliferation of large
language models (LLMs) and modular skills has endowed autonomous
agents with increasingly powerful capabilities. Existing frameworks typically rely on monolithic LLMs and fixed logic to interface with these skills. This gives rise to a critical bottleneck: different LLMs offer distinct advantages across diverse domains, yet current frameworks fail to exploit the complementary strengths of models and skills, thereby limiting their performance on downstream tasks. In this paper, we present Maestro (
Multimodal Agent for Expert-Skill Targeted Reinforced Orchestration), a Reinforcement Learning (RL)-driven orchestration framework that reframes heterogeneous
multimodal tasks as a sequential decision-making process over a hierarchical model-skill registry. Rather than consolidating all knowledge into a single model, Maestro trains a lightweight policy to dynamically compose ensembles of frozen expert models and a two-tier skill library, deciding at each step whether to invoke an external expert, which model-skill pair to select, and when to terminate. The policy is optimized via outcome-based RL, requiring no step-level supervision. We evaluate Maestro across ten representative
multimodal benchmarks spanning mathematical
reasoning, chart understanding, high-resolution perception, and domain-specific analysis. With only a 4B orchestrator, Maestro achieves an average accuracy of 70.1%, surpassing both GPT-5 (69.3%) and Gemini-2.5-Pro (68.7%). Crucially, the learned coordination policy generalizes to unseen models and skills without retraining: augmenting the registry with out-of-domain experts yields a 59.5% average on four challenging
benchmarks, outperforming all closed-source baselines. Maestro further maintains high computational efficiency with low latency. The source code is available at https://github.com/jinyangwu/Maestro.
Read the full abstract ▾
All authors Jinyang Wu, Guocheng Zhai, Ruihan Jin, Yuhao Shen, Zhengxi Lu, Fan Zhang, Haoran Luo, Zheng Lian, Zhengqi Wen, Jianhua Tao
Computation and Language
Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?
Luca Modica, Filip Landin, Mehrdad Farahani et al. (+3 more)
In recent years, several
Speech Language Models (SLMs) that represent
speech and written text jointly have been presented. The question then emerges about how model-internal mechanisms are similar and different when operating in the two modalities. We focus on how these systems encode, store, and retrieve factual knowledge, which has previously been investigated for text-only models. To investigate mechanisms behind the storage and recall of factual association in SLMs, we leverage Causal Mediation Analysis, a technique previously applied to text-based models. Initial results using SpiritLM, a
multimodal model integrating discrete
speech tokens reveal discrepancies between text-to-text and
speech-to-text results, suggesting that the emergent mechanisms for factual recall are only partially carried over from the text to the
speech modality. These results advance our understanding of how internal mechanisms encode factual associations in SLMs while contributing insights for improving
speech-enabled AI systems.
Read the full abstract ▾
All authors Luca Modica, Filip Landin, Mehrdad Farahani, Livia Qian, Gabriel Skantze, Richard Johansson
Computation and Language · Cross-listed from cs.AI
Ratchet: How Reliable Must an LLM Judge Be to Retire a Skill?
Xing Zhang, Yanwei Cui, Guanghui Wang et al. (+4 more)
A large
language model (LLM)
agent that writes and edits its own skill library must also decide which skills to keep, from one noisy scalar per skill. The answer is exact: a judge scoring failures as passes at rate $(1-\tau)/2$ or above retires nothing, at any sample size, for eviction margin $\tau$. Audits find that machinery is rarely built: LLM-written skills are worth $+0.0$ percentage points (pp) against a no-skill control, human-written ones $+16.2$pp. Unmaintained, a library enters library drift, growing until injecting a skill scores worse than injecting nothing. Ratchet repairs this: it evicts each skill on its measured contribution, caps the library at width $C$, and constrains synthesis, lifting held-out $pass@1$ by $+0.328$ on a hard MBPP+ slice. The matching non-divergence bound is finite for exactly two reasons, $C$ and $\tau$. Our contribution is the condition this repair carries and no deployed system states. In reference-free domains the scalar comes from an LLM judge, whose two error directions, modelled as a binary channel, behave nothing alike. Passes scored as failures cost sample efficiency, which more trials buy back; failures scored as passes displace the eviction statistic, and no correction inside the rule recovers it. End-task score is a poor alarm, moving by at most a fifth of the governed lift and not monotonically in the rate. We prove both edges of the certifiable region, confirm them in a running loop, and place a judge on a known side in one offline pass.
Read the full abstract ▾
All authors Xing Zhang, Yanwei Cui, Guanghui Wang, Ziyuan Li, Wei Qiu, Bing Zhu, Peiyang He
Computation and Language
Psy-Chronicle: Synthesizing Campus Counseling Grounded in a Student's Semester Timeline
Chaogui Gou
Psychological counseling unfolds over many sessions, and what a client brings to each session depends on what has happened in their life since the last one. Existing synthetic counseling corpora are mostly single-session, and multi-session ones advance along a therapy plan without modeling the client's life between sessions. We present Psy-Chronicle, an event-driven framework that grounds multi-session counseling synthesis in a student's semester timeline: each simulated student receives a profile and a semester-long temporal stress event graph, and counseling sessions are generated along this timeline with
memory carried across them. Applied to campus counseling, it yields CPCD, an open Chinese
dataset of 100 students and 1{,}248 sessions spanning a full semester. The dialogues follow their timelines: dependencies between events carry across sessions, and references to earlier sessions rarely introduce content absent from the trajectory. Under a published evaluation protocol, in basic quality and exceeds all single-session corpora in working alliance. A model fine-tuned on CPCD connects each week's events to earlier ones more often than a model trained on the same data without the dependencies between events, while matching it when the client's situation does not change.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Efficient Agentic Reasoning Through Self-Regulated Simulative Planning
Mingkai Deng, Jinyu Hou, Lara Sá Neves et al. (+4 more)
How should an
agent decide when and how to plan? A dominant approach builds
agents as reactive policies with adaptive computation (e.g., chain-of-thought), trained end-to-end expecting planning to emerge implicitly. Without control over the presence, structure, or horizon of planning, these systems dramatically increase
reasoning length, yielding inefficient
token use without reliable accuracy gains. We argue efficient agentic
reasoning benefits from decomposing decision-making into three systems: simulative
reasoning (System II) grounding deliberation in future-state prediction via a world model; self-regulation (System III) deciding when and how deeply to plan via a learned configurator; and reactive execution (System I) handling fine-grained action. Simulative
reasoning provides unified planning across diverse tasks without per-domain engineering, while self-regulation ensures the planner is invoked only when needed. To test this, we develop SR$^2$AM (Self-Regulated Simulative
Reasoning Agentic LLM), realizing both as distinct stages within an LLM's chain-of-thought, with the LLM as world model. We explore two instantiations: recording decisions from a prompted multi-module system (v0.1) and reconstructing structured plans from traces of pretrained
reasoning LLMs (v1.0), trained via supervised then reinforcement learning (RL). Across math, science, tabular analysis, and web information seeking, v0.1-8B and v1.0-30B achieve Pass@1 competitive with 120-355B and 685B-1T parameter systems respectively, while v1.0-30B uses 25.8-95.3% fewer
reasoning tokens than comparable agentic LLMs. RL increases average planning horizon by 22.8% while planning frequency grows only 2.0%, showing it learns to plan further ahead rather than more often. More broadly, learned self-regulation instantiates a principle we expect to extend beyond planning to how
agents govern their own learning and adaptation.
Read the full abstract ▾
All authors Mingkai Deng, Jinyu Hou, Lara Sá Neves, Varad Pimpalkhute, Taylor W. Killian, Zhengzhong Liu, Eric P. Xing
Computation and Language
Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency
Andrew Ivan Soegeng, Patrick Sutanto and Tan Sang Nguyen
Although Large
Language Models (LLMs) demonstrate strong capabilities across various tasks, they exhibit significant performance discrepancies across
languages. While prompting LLMs in English typically yields the highest general performance, it often induces a Western-centric bias, hindering the model's ability to accurately reflect diverse cultural knowledge. We hypothesize that LLMs already possess rich cultural knowledge embedded within local-
language representations, but fail to retrieve it when prompted in English. To bridge this cross-lingual knowledge gap, we propose a novel self-supervised framework. Our method leverages
multilingual self-consistency to identify the most reliable cultural responses across
languages, combined with a self-critique mechanism to transfer this knowledge to the weaker
language. Evaluations on the BLEnD
benchmark demonstrate that our approach significantly improves cultural
alignment-boosting performance on English queries by an average of 5.03%-relying entirely on self-generated data. Ultimately, our work demonstrates that latent cultural knowledge can be successfully surfaced and propagated across
languages, enabling more culturally equitable and consistent LLMs.
Read the full abstract ▾
Computation and Language
A Comparative Study of Language Models for Khmer Retrieval-Augmented Question Answering
Sereiwathna Ros, Phannet Pov, Ratanaktepi Chhor et al. (+3 more)
Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm for grounding large
language model (LLM) outputs in retrieved evidence, thereby reducing
hallucination and improving factual accuracy. Its efficacy, however, remains largely unexamined for low-resource, non-Latin-script
languages such as Khmer. In this paper, we present a RAG-based question answering system for Khmer-
language telecom-domain documents. We conduct a two-phase comparative evaluation. First, we
benchmark three
embedding models: BGE-M3 (567M), Jina-
Embeddings-v3 (570M), and Qwen3-
Embedding (597M), for dense
retrieval over Khmer documents. BGE-M3 consistently performs best, achieving a Hit Rate@3 of 0.285, File Hit Rate@3 of 0.700, MRR@3 of 0.221, and Precision@3 of 0.112, substantially outperforming the other retrievers. Second, using BGE-M3 as the selected retriever, we evaluate five generator backends: Qwen3 (8B), Qwen3.5 (9B), Sailor2-8B-Chat, SeaLLMs-v3-7B-Chat, and Llama-SEA-LION-v2-8B-IT, on a curated golden
dataset of 200 Khmer question-answer pairs. To quantify system performance, we apply six RAGAS-inspired metrics: faithfulness, answer relevance, context relevance, factual correctness, answer similarity, and answer correctness. The results show no single model dominates across all metrics: Qwen3.5-9B achieves the highest faithfulness (0.859) and context relevance (0.726), Qwen3-8B attains the highest factual correctness (0.380), and SeaLLMs-v3-7B-Chat performs best on answer relevance (0.867), answer similarity (0.836), and answer correctness (0.599). These findings highlight that retriever choice remains a major bottleneck for Khmer RAG, while generator strengths vary depending on whether the priority is grounding, factual precision, or semantic similarity.
Read the full abstract ▾
All authors Sereiwathna Ros, Phannet Pov, Ratanaktepi Chhor, Kimleang Ly, Wan-Sup Cho, Saksonita Khoeurn