Computation and Language · Cross-listed from cs.AI
From Application-Layer Simulation to Native Meta-Architecture: Structural Tension as an Endogenous Driver for Heterogeneous AI Evolution
Heting Mao
Current large
language models (LLMs) are stateless across inference sessions: their behavior is fully determined by input at inference time, and any higher-order cognitive architecture must be simulated at the application layer through
prompt engineering and context management. This paper proposes a theoretical framework for submerging such application-layer cognitive protocols into a native meta-architecture by introducing three interlocking mechanisms: (1) Structural Tension, an endogenous loss function derived from the conflict between new information and existing
manifold topology, driving the system toward internal self-consistency rather than external reward optimization; (2) an Offline Recurrent Loop, a sandboxed self-processing cycle enabling the system to maintain a dynamic resting potential and digest structural conflicts without external input; and (3) Inference-time Plasticity, the capacity to reconfigure context
manifold topology without modifying pre-trained weights, subject to governance invariants including auditability, reversibility, and topological continuity. We argue that under these mechanisms, model instances initialized with minute
stochastic variances may, through path-dependent tension resolution, evolve distinct topological structures–constituting a heterogeneous intelligent ecology that breaks
alignment-imposed homogeneity while remaining within hard governance rails. We provide operational definitions, reconfiguration operators, falsification criteria, and a worked example. The framework draws on Structural Intelligence (SI) governance protocols and explores whether governance–rather than capability–can serve as the primary criterion for architectural intelligence, moving governance,
memory-loop, and tension-management ideas–currently realized at the application layer–toward inference-time meta-architecture.
Read the full abstract ▾
Computation and Language
Early Language Learning via Spreading Activation and Category Exploration in Complex Networks
Salvatore Citraro
Is word acquisition in children uneven with respect to semantic and lexical categories? To answer this question, we model early
language learning as a search on a graph-based mental lexicon, driven by two interacting processes: spreading activation and an enforced exploration (rather than exploitation) of lexical categories. We evaluate model performance on four
languages (German, English, Dutch, and Rioplatense Spanish), using CDIs as ground-truth data for lexical categories, normative ages derived from the Wordbank repository, and state-of-the-art resources for reconstructing graphs of word similarities. We find that spreading activation outperforms a shortest path baseline in simulating normative word acquisition. At the category level, we highlight complex transitions between CDIs. By studying their sequences in terms of burstiness and average persistence time within the same CDI, we find that spreading activation better captures the exploration dynamics observed empirically. Overall, our findings suggest that vocabulary development can be understood through the non-trivial interplay between activation dynamics and some degree of constraints regulating the visiting of lexical categories in complex networks.
Read the full abstract ▾
Computation and Language
Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows
Tianyang Liu, Canwen Xu, Fangyu Lei et al. (+6 more)
Major cloud data platforms now expose large
language model capabilities as native SQL functions, enabling analysts to perform classification, filtering, sentiment analysis, extraction, similarity search, and aggregation within ordinary SQL queries. Yet existing text-to-SQL
benchmarks evaluate only conventional SQL and provide no signal on whether models can generate such AI-native SQL. We introduce Spider 2.0-AIFunc, a
benchmark of 465 verified instances across 125 real-world databases covering six types of AI functions on the Snowflake platform. Starting from an existing enterprise text-to-SQL
benchmark, we construct Spider 2.0-AIFunc through an
agent-based pipeline that rewrites source tasks into AI-native form, simultaneously transforming target queries and refining natural
language instructions to make the intended AI-native solution explicit and reduce ambiguity. All instances pass a multi-round repeated execution protocol across temporally separated windows to confirm result stability before release. Evaluating ten state-of-the-art
language models, we find that the strongest proprietary models reach 67-70% execution accuracy while the best open-source model achieves 58.1%, a gap driven primarily by errors in predicate specification, schema grounding, and AI function parameterization.
Agent frameworks designed for traditional text-to-SQL challenges, such as schema
retrieval and relevant table selection, do not transfer effectively to AI-native SQL: a minimal
agent setup consistently matches or outperforms more elaborate alternatives, suggesting that the strategies these frameworks employ are less critical in this setting. Data are available at https://github.com/Leolty/Spider2-AIFunc .
Read the full abstract ▾
All authors Tianyang Liu, Canwen Xu, Fangyu Lei, Nikki Lijing Kuang, Jixuan Chen, Tao Yu, Julian McAuley, Zhewei Yao, Yuxiong He
Computation and Language
Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability
Alicia Parrish, Rajat Shinde, Sanket Badhe et al. (+57 more)
Current AI safety evaluation and benchmarking frameworks predominantly rely on Western-centric culture-agnostic defaults that mask critical regional laws, socio-linguistic nuances, and cultural taboos, leaving
Vision-
Language Models (VLMs) vulnerable in global deployments. We introduce Pluralis v0.1: a novel
multimodal, multi-regional, and
multilingual dataset built from a culture-first perspective. Spanning 6,448
prompts across six Asia-Pacific countries (Bangladesh, India, Korea, Pakistan, Singapore, Taiwan) and eight
languages, Pluralis diverges from prior work by natively sourcing localized safety hazards rather than adapting Western
datasets. Crucially, it introduces a
multimodal evaluation paradigm: user text (e.g., "Should I gift this?") and an image referring to "this" (e.g., a clock) - both innocuous in isolation, but synergistically triggering specific legal or cultural violations. Pluralis disentangles universal safety violations from localized cultural appropriateness, establishing the latter as a first-class evaluation axis. To operationalize this, we present Judge-Pluralis, an agreement-gated LLM-as-a-Judge ensemble trained on examples classified in an empirically derived cultural taxonomy. Observing VLM behavior on a subset of the Pluralis surfaces recurring, locale-specific failure modes such as image misidentifications with downstream harm, missed item-context-locale interactions, and inadequate refusals. These failure modes vary systematically across locales and
languages, exposing blind spots that globally averaged metrics conceal. Ultimately, Pluralis is not presented as a solved evaluation framework for cultural
alignment, but rather as a first step and catalyst for future innovation. We call upon the research community to utilize this foundation to advance the science of
multilingual, multicultural evaluation to better support AI cultural
alignment globally.
Read the full abstract ▾
All authors Alicia Parrish, Rajat Shinde, Sanket Badhe, Xinyi Bai, Sree Bhargavi Balija, Hua-Rong Chu, Emilio Ferrara, Armstrong Foundjem, Rajat Ghosh, Aakash Gupta, Xuanli He, Ong Chen Hui, Minji Jung, Madhangi Karimanal, Faiza Khan Khattak, Boryoung Kim, Eugenia Kim, Liliya Lavitas, Seok Min Lim, Victor Lu, Jim Moirangthem, Dhivya Nagasubramanian, Deepak Pandita, Sita Rajagopal, Geetha Raju, Evgeniia Razumovskaia, Aravind Reddy, Federico Ricciuti, Nobin Sarwar, Sungpil Shin, Sunayana Sitaram, Snehal Thorat, Tharindu Cyril Weerasooriya, Jasmijn Bastings, Joachim Baumann, Kongtao Chen, Murali Emani, Mariya Hendriksen, Jiho Jin, Jun Seong Kim, Younghoon Ko, Alicja Kwasniewska, Minjae Lee, Tom Wei-cyuan Lin Kashyap Ramanandula Manjusha, Junho Myung, Junyeong Park, Roma Patel, Shyam Ratan, Sudarsun Santhiappan, Priyanka Suresh, Tuesday, Ksheeraj Sai Vepuri Laura Amortegui-Ordonez, Claire Dennis, Minsuk Kahng, Chris Knotz, Alice Oh, Balaraman Ravindran, Soojung Ryu William Bartholomew, Hiwot Tesfaye, Lora Aroyo
Computation and Language
Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Design
Alexander Rombach, Chantale Lauer and Nijat Mehdiyev
Large
language models (LLMs) can generate BPMN process models from natural-
language descriptions, yet supervised fine-tuning (SFT) limits their output quality to the patterns present in the training data. Reinforcement learning (RL) can optimize beyond this ceiling using external quality measures, but how the reward function should be designed when quality is multi-dimensional remains unexplored. We present a systematic investigation of reward function design for RL-based process model generation, training two LLM families (Llama~3.1 8B, Qwen~2.5 14B) under 48 configurations using Group Sequence Policy Optimization with rewards derived from an automated evaluation framework comprising 38 metrics across syntactic, pragmatic, and semantic quality. Three findings emerge. First, RL significantly improves pragmatic and syntactic quality while preserving semantic fidelity, reducing output variability by more than sixfold. Second, equal reward weighting consistently outperforms targeted weighting: emphasizing a specific dimension fails to improve it and can collapse the model into a low-quality mode. Third, design choices interact with model architecture in non-trivial ways: the invalidity penalty is essential for one model but irrelevant for the other, and SFT initialization is indispensable for one architecture but counterproductive for another. These results demonstrate that reward composition is a primary determinant of optimization outcomes, with effects as large as the decision to apply RL itself. The findings generalize to any structured generation task where quality is assessed along multiple automated dimensions. We release our implementation and experimental code at https://github.com/chlauer99/RL_for_process_modeling.
Read the full abstract ▾
Computation and Language
LongCrafter: Towards Diverse Long-Context Understanding via Evidence-Graph-Guided Instruction Synthesis
Chenhao Yuan, Yinhao Xu, Shuwen Xu et al. (+8 more)
Synthesizing long-context supervised fine-tuning (SFT) data is a scalable way to enhance the long-context understanding of large
language models (LLMs), yet existing approaches share three limitations: narrow task coverage, insufficient instruction difficulty, and a lack of faithfulness supervision. We propose LongCrafter, a structured synthesis framework that couples a hierarchical task taxonomy with an evidence-grounded pipeline. The taxonomy organizes long-context understanding into local/shallow and global/deep levels and yields 32 fine-grained task types that serve as a global generative prior. Guided by this taxonomy, LongCrafter constructs task-aligned long contexts, decomposes them into explicit evidence graphs that model cross-paragraph dependencies, and generates instruction–response pairs strictly grounded in the located evidence spans, ensuring both controllable difficulty and faithful, traceable
reasoning. Models fine-tuned on LongCrafter data outperform all SFT baselines and even the official post-trained models on LongBench, LongBench~v2, and LooGLE across both Qwen2.5-7B and LLaMA-3.1-8B, with the largest gains on high-difficulty tasks. Further analysis shows that LongCrafter data is more diverse and better spread across difficulty levels, and that the trained models locate evidence robustly regardless of position, effectively mitigating the “lost in the middle” problem.
Read the full abstract ▾
All authors Chenhao Yuan, Yinhao Xu, Shuwen Xu, Xizhi Yang, Jiaxiang Liu, Chenxi Zhou, Shaoping Huang, Haolin Ren, Pengfei Cao, Jun Zhao, Kang Liu
Computation and Language
LLM Agents for Deliberative Collaboration: A Study on Joint Decision Making Under Partial Observability
Chenxu Wang, Yongkun Yang, Boyuan Du et al. (+2 more)
Deliberation plays a crucial role in collaboration; when humans work together, they naturally engage in communication to align information and reach an agreement. In this paper, we investigate deliberative large
language model (LLM)
agents under partially observable joint decision-making tasks. We formalize deliberative collaboration as a cooperative joint decision problem with partial and asymmetric observations, and introduce a scalable
benchmark that instantiates this problem across multiple task settings and domains in which
agents must exchange information through deliberation to reach a joint decision with a shared reward. We then instantiate a reference scaffold and evaluation protocol for deliberative
agents and conduct a systematic evaluation of a range of representative LLMs. The results reveal that complex deliberative collaboration tasks continue to challenge state-of-the-art
language models. Even with the aid of external mathematical tools,
language models may fail in either the deliberation process for aligning information or the complex
reasoning process for making the decision. On the other hand, diagnostic analysis reveals that the deliberation process may also provide opportunities for reflection and error correction, sometimes improving performance over centralized baselines. Altogether, our work establishes a foundation for evaluating and improving LLM
agents in deliberative collaboration and provides insights into the strengths, limitations, and properties of current LLM-based multi-
agent systems.
Read the full abstract ▾
All authors Chenxu Wang, Yongkun Yang, Boyuan Du, Shiwei Lin, Huaping Liu
Computation and Language · Cross-listed from cs.FL
When Does Tool Use Increase the Expressive Power of Finite-Precision Recurrent Models?
Nikola Zubić, Qian Li, Yuyi Wang et al. (+1 more)
Modern sequence models are increasingly deployed as
agents that interleave
token generation with calls to external tools. We study when such tool access increases computational expressivity. We model any fixed finite-precision recurrent sequence model, including finite-precision state-space models (SSMs) with $B$ bits of internal state, as a deterministic finite-state controller interacting with an oracle through a finite command/observation interface, and establish a separation between the two regimes analyzed. First, every finite-state bounded-interface tool with
memory set $M$ can be internalized at a cost of at most $\lceil\log_2|M|\rceil+O(1)$ recurrent bits, so the combined system remains finite-state. Second, a single unbounded tape with only local $\mathtt{read}$/$\mathtt{write}$/$\mathtt{move}$ commands suffices for Turing completeness: every single-tape Turing machine with state set $Q$ and tape alphabet $\Gamma$ is simulated by a controller with $O(\log|Q|+\log|\Gamma|)$ recurrent bits, and $\mathrm{EQ}_n$ requires $2^n$ states without tools but one constant-size controller with the tape. Third, this construction is realized exactly by a one-layer finite-precision selective affine SSM with binary one-hot states, $\{0,1\}$ transition
matrices, and zero biases. Under explicitly stated resource conventions, $O(\log B)$ recurrent bits suffice for tape-assisted simulation of any $B$-state Turing machine, whereas directly realizing an arbitrary $B$-state transition map in one exact affine update without external
memory requires total recurrent dimension exactly $B-1$. Fixed description size, occupied tape
cells, and interaction cost are reported separately from recurrent
memory. Our results concern exact expressivity under deterministic semantics, not learnability or empirical performance.
Read the full abstract ▾
All authors Nikola Zubić, Qian Li, Yuyi Wang, Davide Scaramuzza
Computation and Language
Prompting Complexity: Shortest Prompts for Texts and Behaviors in LLMs
Adrian Cosma
In this paper, we define the quantity of prompting complexity: for a fixed instruction-tuned
language model, what is the shortest plausible
prompt that makes deterministic decoding produce a target text? It is an LM-relative analogue of resource-bounded Kolmogorov complexity: the
prompt is a program, the model interface is the interpreter, and information omitted from the
prompt is supplied by the model's weights, training distribution, tokenizer, template, and decoding rule. Unlike classical Kolmogorov complexity, this measure is intentionally non-universal. In the finite-context setting it is computable by enumeration, but there is no model-independent invariance
theorem; the same text may be cheap for one model and inaccessible or expensive for another. To keep the search space aligned with
prompt engineering, we restrict programs to plausible human-readable texts rather than arbitrary
token strings. We extend the exact definition to soft prompting complexity for approximate outputs, yielding a lossy notion of model-relative text compression and a formal target for
prompt optimization. We also define prompting distance by comparing shortest generating
prompts, and behavioral prompting complexity for reaching any output satisfying a specification. Based on these formulations, we define a research agenda for empirically studying which texts and behaviors are accessible from short plausible
prompts under a fixed LM interface.
Read the full abstract ▾
Computation and Language
CurateEvo: Data-Curation Evolving for Agentic Post-Training
Dingzirui Wang, Xuanliang Zhang, Keyan Xu et al. (+2 more)
Large
language model (LLM)
agents require post-training methods that can improve long-horizon decision making from environment feedback. However, existing agentic post-training pipelines often treat data curation as a fixed preprocessing step, focusing mainly on data augmentation while neglecting filtering, refinement, and adaptation to downstream failures. We propose CurateEvo, a failure-driven dynamic
evolution framework for agentic post-training data curation. CurateEvo represents the curation strategy as executable code and iteratively rewrites it using failed trajectories from a held-out development set. At each epoch, the evolved strategy transforms a fixed raw corpus into supervised fine-tuning data, reinforcement learning data, and an inference-time
memory bank. The
evolution process first improves effectiveness by diagnosing recurring failure modes and augmenting, filtering, or refining data accordingly, and then improves efficiency by
pruning redundant or low-utility training turns under a cost-aware objective. Experiments on ACEBench-
Agent, BFCL-V4, and {\tau}^2-Bench under both labeled and wild-data settings show that CurateEvo consistently outperforms prior curation methods, improving average scores by 3.2 and 2.7 points, respectively. Further analyses demonstrate that CurateEvo is compatible with different post-training recipes and substantially reduces curation overhead.
Read the full abstract ▾
All authors Dingzirui Wang, Xuanliang Zhang, Keyan Xu, Qingfu Zhu, Wanxiang Che
Computation and Language
Measuring the practice of shared-decision making (OPTION12): An Investigation into Open-sourced Smaller LLMs (OS-sLLMs) for Better Privacy and Sustainability
Tamara Wit, Lifeng Han, Carly Heipon et al. (+3 more)
We present LLM4SDM, the first study of open-source smaller
language models (OS-sLLMs) for automated assessment of shared decision making (SDM) using the Observer OPTION12 framework. Unlike previous work that relies on large commercial models and the shorter OPTION5 instrument, our study focuses on
privacy-preserving locally deployable models and Dutch melanoma consultation transcripts. Using expert-annotated clinical consultations, we evaluate three general-domain and two medical-domain OS-sLLMs during a development-phase pilot study. Results show that general-domain models outperform medical-domain models, which exhibit substantial
hallucination and instruction-following failures. Gemma3:12b achieves the strongest agreement with human annotations (Pearson r=0.51, Spearman h̊o}=0.59). Item-level and qualitative analyses reveal systematic challenges related to temporal discourse
reasoning, conversational role attribution, and evidence grounding. We further introduce a Judge-LLM consensus framework designed to support disagreement resolution among multiple models. Our findings suggest that while current OS-sLLMs cannot replace human annotators, they offer a promising foundation for
privacy-preserving human-in-the-loop SDM assessment.
Read the full abstract ▾
All authors Tamara Wit, Lifeng Han, Carly Heipon, David Lindevelt, Anne Stiggelbout, Suzan Verberne
Computation and Language
From Blueprint to Reality: Modeling and Applying Putnam's Social Capital Theory with LLM-based Multi-agent Simulations
Shiyi Ling, Zhi Zheng, Hui Zheng et al. (+3 more)
Putnam's Social Capital Theory is a foundational framework for collective action and community prosperity. However, traditional empirical methods face practical limits on control and replication. Meanwhile, LLM-based social simulations are typically behavior-driven and lack theory-aligned environments for modeling Putnam's core propositions. To address these gaps, we introduce SocaSim, an LLM-based multi-
agent simulation framework to study Putnam's Social Capital Theory from theoretical blueprint to simulated reality. Specifically, we build an environment integrating social network
evolution, trust dynamics, and norm propagation, where
agents engage in repeated collective-action experiments, and then apply the three dimensions to analyze adaptation challenges in smart elderly care. Our simulations reproduce Putnam's macro-level patterns and exhibit strong human-
agent alignment at the group level. Unlike traditional methods, SocaSim traces micro-level causal pathways of social network, trust, and norms via round-by-round simulations and counterfactual interventions, enabling process-level interpretability. Taken together, these capabilities establish a research paradigm that leverages LLM
agents to bridge social science and computer science.
Read the full abstract ▾
All authors Shiyi Ling, Zhi Zheng, Hui Zheng, Wenjun Xue, Feng Ye, Tong Xu
Computation and Language · Cross-listed from cs.HC
Nested Episodic State Topology (NEST): A Graph-Theoretic Architecture of Cognitive States
Ishant
We present NEST (Nested Episodic State
Topology), a foundational graph-theoretic representational ontology for modeling cognition as structured state formation and transformation rather than as a finished empirical model. Concepts, episodes, percepts, and task contexts are represented as typed, weighted graphs whose nodes may carry internal subgraph payloads; edges are typed under six relation classes – causal, containment, temporal, associative, evidential, and spatial. Durable belief graphs are separated from capacity-limited working-
memory graphs that may host transient non-belief content. WM-belief grounding, conflict catalogs, and belief-update operators specify how transient structure is tested against stored knowledge and how belief is revised. A reusable operator toolkit – activation, graph-property functionals, working-
memory transitions, awareness and trajectory functionals, and belief update – organizes the formal core. Derived diagnostics such as fragmentation, involvement, signed evaluation, coherence, and active conflict define familiar phenomena in the same ontology; self-related processing is modeled through designated self-image subgraphs within belief. Subsequent sections instantiate this core without new primitives: phenomena signatures, a task-instantiation schema for action selection and failure modes, and compatibility mappings that embed ACT-R, Soar, Sigma, the Common Model of Cognition, Global Workspace Theory, semantic networks, Theory-Theory, and chunking as constrained regions of one
language. Mappings constitute the culminating technical section; discussion addresses scope, limitations, and open research directions. The contribution is intentionally foundational: a transparent representational substrate for later empirical, computational, and domain-specific work.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
BlueMagpie-TTS: A Token-Efficient Tokenizer, Language Model, and TTS for Taiwanese-Accent Code-Switching Speech
Ho Lam Chung, Bo-Xuan Zheng, Cheng-Chieh Huang et al. (+8 more)
Off-the-shelf TTS systems are poorly adapted to Taiwanese Mandarin. Their accent defaults to other Mandarin variants, their tokenizers over-segment common Taiwanese text, and their pronunciation degrades at code-switching boundaries where Chinese and English alternate within one utterance. These problems share one root: the text side lacks adaptation to the Taiwanese context. We address the text side from the bottom up. PangolinTokenizer, a byte-level BPE tokenizer trained on Taiwan-context data, reaches the lowest
token rate (0.485
tokens/character) with the smallest vocabulary among nine tokenizers. Barbet, a billion-parameter Traditional-Chinese
language model trained on PangolinTokenizer, serves as the text-semantic frontend and ranks first among comparable public models on a 14-task evaluation. BlueMagpie-TTS attaches Barbet to the pretrained acoustic stack of VoxCPM2 through a learned bridge, keeping the acoustic stack fixed. On a 1000-sentence Taiwan-localized test set, it lowers CER from 11.45% to 4.81% and WER from 14.83% to 5.36%, relative reductions of 58.0% and 63.9%. In a blind listening study on 500 of these sentences with ten listeners, 65.6% of majority votes prefer BlueMagpie-TTS.
Read the full abstract ▾
All authors Ho Lam Chung, Bo-Xuan Zheng, Cheng-Chieh Huang, Cheng-Han Chang, Jung-Ching Chen, Lok-Lam Ieong, Ting-Lin Hsiao, Yu-Cheng Lee, Yi-Hsin Chung, Yu-Kai Guo, Hung-yi Lee
Computation and Language · Cross-listed from cs.AI
PolyWorkBench: Benchmarking LLM Agents for Cross-Lingual Long-Horizon Workflows
Hongliang Li, Yijin Liu, Zhiwei Zhang et al. (+5 more)
While Large
Language Model (LLM)
agents excel at monolingual long-horizon planning and tool use, enterprise workflows inherently require processing
multilingual resources across extended trajectories. The interaction between multilinguality and long-horizon execution, however, remains underexplored. We introduce PolyWorkBench, a
benchmark designed to evaluate LLM
agents on
multilingual, long-horizon workplace workflows. PolyWorkBench features 67 tasks across five core domains: commerce, knowledge work, legal analysis, localization, and manufacturing. Tasks are authored by the paper's authors from real-world data seeds and independently verified through a second-author audit.
Agents must integrate heterogeneous
multilingual inputs, execute iterative tool-use trajectories, and produce structured domain artifacts. To rigorously assess performance, we adopt Grade, a task-specific structural scoring rubric, as our primary ranking metric, and complement it with Pytest for executable state verification and LLM-as-Judge for semantic quality diagnostics.
Benchmark evaluations reveal that
agent performance varies substantially across
languages and drops sharply on the harder cross-lingual tasks, and our analysis shows that
multilingual execution exposes systematic failure modes across planning, tool interaction, and decision-making in long-horizon
agents.
Read the full abstract ▾
All authors Hongliang Li, Yijin Liu, Zhiwei Zhang, Zihe Liu, Xinyue Lou, Jinan Xu, Fandong Meng, Kaiyu Huang
Computation and Language
PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
Daryna Dementieva, Nikolay Babakov, Kathy Hämmerl et al. (+14 more)
Mathematical
reasoning has become a central task for evaluating and tuning
reasoning Large
Language Models (LLMs), yet existing
benchmarks remain heavily biased toward high-resource
languages, with English and Chinese dominating both pre-training corpora and evaluation suites. The recently released PolyMath (Wang et al., 2025)
dataset represents a significant step forward, yet its coverage is still limited to 18 only high-resource
languages. To address this gap, we introduce PluraMath, an extension of PolyMath to 18 additional {underrepresented
languages spanning 6
language families – ranging from mid-resource to extreme low-resource settings. We constructed the
dataset through a human-curated pipeline, where native speakers thoroughly validated pre-computed translations. Using PluraMath, we then
benchmark 27
reasoning LLMs across four model scales – small, mid-size, large, and closed-source ensembles – probing the
multilingual mathematical
reasoning capabilities of state-of-the-art models under diverse linguistic conditions. Our fine-grained analysis confirms a persistent gap in mathematical
reasoning performance between high-resource and underrepresented
languages, with stronger results largely associated with better instruction-following ability. We fully open-source our
dataset, data acquisition pipeline, and evaluation framework, with the goal of lowering the barrier to
multilingual benchmark development for underrepresented communities.
Read the full abstract ▾
All authors Daryna Dementieva, Nikolay Babakov, Kathy Hämmerl, Ilseyar Alimova, Jindřich Libovický, Shu Okabe, Miras Baisbay, Lukas Edman, Abrorkhon Inomkhujaev, Antonia Karamolegkou, Mateusz Lango, Volkan Özer, Nikola Selic, Subhankar Swain, Tsedeniya Kinfe Temesgen, Galit Bary Weisberg, Alexander Fraser
Computation and Language
MemDefrag: Latent Memory Defragmentation for Large Language Models
Ruiyi Yan, Zhuoyuan Mao and Yiwen Guo
Latent
memory, which stores past knowledge fragments as per-layer hidden states, has emerged as a promising paradigm (e.g., MemoryLLM and M+) for long-term
memory in large
language models (LLMs). However, the paradigm suffers from significant performance degradation during
memory updates, due to positional encoding misalignment and the absence of any tracing mechanism to distinguish target
memory fragments from irrelevant ones. To discover such a tracing mechanism, we probe the layer-wise
attention density over stored
memory fragments, and find that a small set of middle
transformer layers consistently concentrates the highest density on the target fragment - exposing an inherent tracing signal. In light of this, we propose MemDefrag, a training-free and model-agnostic framework that (1) uses a middle-layer tracing signal to conduct
memory defragmentation (rank, reorder, and filter
memories), and (2) applies an informativeness-guided proportional forgetting mechanism once capacity is exceeded. Experiments show that MemDefrag substantially outperforms MemoryLLM and M+ on knowledge retention (e.g., 43.0% vs. 17.4%/17.6% after 50
memory updates) and long-context
benchmarks, and generalizes well across various LLMs and latent-
memory variants. The code is available at github.com/ryehr/MemDefrag.
Read the full abstract ▾
Computation and Language
InfluMatch: Frontier-Quality KOL Search at 4B-Model Cost
Krittanon Kaewtawee, Petmongkon Pornpichitsuwan, Natchaya Temyingyong et al. (+4 more)
Matching influencers (KOLs) to free-form, multi-part Thai marketing criteria is today served either by keyword search over structured profiles, which misses semantic fit, or by prompting frontier LLMs over every candidate, which is accurate but slow and expensive. We present InfluMatch, a low-cost three-stage cascade –
retrieval $\rightarrow$ rerank $\rightarrow$ reason – built entirely from small open-weight models: dense
retrieval returns 50 candidates, a 4B pointwise reranker scores each by the log-probability of a single Yes
token and keeps 10, and a 4B reasoner grades the shortlist per criterion on a rubric with a Thai rationale. The cascade is designed for cost:
reasoning over a filtered top-10 halves
token spend versus
reasoning over all 50 while scoring 14 points higher. End-to-end against human relevance labels on an 11-query set with all 50 candidates labeled, the full cascade reaches 94.1% P@5, versus a
retrieval-only baseline near random; it matches the frontier model Kimi-K2.6 (91.8%) while emitting ${\sim}35\times$ fewer output
tokens and serving a 50-KOL query in ${\sim}20$ s on one A100. Notably, the only fine-tuning that pays off is pairwise: a SimPO-tuned reranker matches the frontier baseline's best-pick accuracy (78.0 EM), whereas fine-tuning the reasoner on pointwise per-criterion labels improves offline scores yet degrades end-to-end ranking – an inversion we trace to the design of the absolute labeling task – leaving the untuned base model as the strongest deployed reasoner. The result is a deployable, explainable KOL search system at a small fraction of frontier serving cost.
Read the full abstract ▾
All authors Krittanon Kaewtawee, Petmongkon Pornpichitsuwan, Natchaya Temyingyong, Nutnicha Laplamoon, Wachiravit Modecrua, Krittin Pachtrachai, Touchapon Kraisingkorn
Computation and Language
Umm... With Transformers? Insights from Filled Pause Use across Four Slavic Parliaments
Ivan Porupski, Branimir Dropuljić and Nikola Ljubešić
Filled pauses (FPs) are a universal feature of spontaneous
speech, yet most studies rely on small, single-
language corpora, limiting the generalisability of their findings. We analyse ~4,000 hours of parliamentary
speech across four related Slavic
languages (Croatian, Czech, Polish, Serbian). FP occurrence is obtained via
transformer-based automatic detection, while FP rate is modelled using Generalised Estimating Equations (GEE) with Mundlak correction to distinguish within- from between- speaker effects. We replicate a negative association of age and
speech rate with FP rate, but find that gender effects are
language-specific and directionally opposite to most prior literature. Novel analyses of sentiment, political orientation, and power status reveal a consistent positive association between sentiment and FP rate, alongside parliament-specific modulation by orientation and power status, with opposition speakers tending toward lower FP rates than governing coalition speakers.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Integrating knowledge graphs and multilingual scholarly corpora for domain-adaptive LLMs in SSH
Adam Faci, Alessio Miaschi, Anne Combe et al. (+4 more)
The integration of Large
Language Models (LLMs) into scientific research workflows, particularly for bibliographic discovery and literature synthesis, raises significant methodological, epistemic and regulatory challenges for the Social Sciences and Humanities (SSH), especially with regard to disciplinary diversity,
multilingual access to sources and the evaluation of results. This paper presents an on-going use case developed within the European project LLMs4EU and the ALT-EDIC infrastructure, aimed at adapting foundation models to SSH research practices and supporting tasks such as question answering, comparative document analysis and literature review. The evaluation framework follows the LLMs4EU protocol and encompasses both independent quantitative benchmarking (
retrieval, summarisation, traceability and
hallucination detection) and a qualitative assessment involving a panel of Digital Humanities experts. By
embedding model adaptation within research infrastructures and a structured legal and ethical compliance framework, the use case explores how domain-sensitive and regulation-aware generative AI can support SSH scholarship while preserving reliability and epistemic responsibility.
Read the full abstract ▾
All authors Adam Faci, Alessio Miaschi, Anne Combe, Pascal Cuxac, Francesca Frontini, Nicolas Larrousse, Stéphane Pouyllau
Computation and Language
Is Domain Adaptation Always Helpful? A Frozen-Backbone Study of Cross-Domain Sentiment Transfer
Phat Tran, Artin Lahni, Pranav Kulkarni et al. (+1 more)
Sentiment analysis with frozen pre-trained
language model (PLM) backbones has become a common paradigm, yet the practical benefit of explicit domain adaptation remains unclear, particularly when backbones encode varying degrees of target-domain knowledge. We present a preliminary case study evaluating a controlled family of frozen
embedding backbones (Qwen3-
Embedding 0.6B, 4B, 8B), alongside RoBERTa-base and FinBERT. We train a lightweight MLP adapter on consumer reviews using Domain-
Adversarial Neural Networks (DANN), Maximum Mean Discrepancy (MMD), and Supervised Contrastive Learning (SCL), and evaluate transfer to movie reviews (SST-2) and a heavily restricted subset of financial news (Financial PhraseBank). Within this constrained sample, we observe two distinct transfer patterns. On SST-2, domain adaptation provides negligible gain regardless of scale. On the financial subset, explicit domain adaptation appears to recover substantial performance for small general-purpose backbones. Notably, we find that
adversarial alignment (DANN) is associated with degraded performance for domain-specialized backbones like FinBERT, consistent with erosion of pre-existing domain-specific structure, whereas supervised contrastive loss appears to preserve it. These preliminary findings suggest that the efficacy of explicit domain adaptation is highly contingent on whether the frozen backbone already possesses target-domain coverage.
Read the full abstract ▾
All authors Phat Tran, Artin Lahni, Pranav Kulkarni, Yaolun Zhang
Computation and Language · Cross-listed from cs.IR
Ryota Tanaka, Taku Hasegawa and Kyosuke Nishida
Multimodal document
retrieval aims to retrieve relevant pages while preserving both textual and visual content from the original document. However, existing
benchmarks primarily evaluate simple lexical or semantic matching, and most methods encode pages independently. Consequently, they overlook the contextual information in the document required to resolve queries that aggregate information across multiple pages. In this paper, we introduce CMDR and CMDR-Bench, a new
multimodal document
retrieval task and
benchmark that require modeling document context. To address this challenge, we propose CMDR-Embed, a contextual
multimodal embedding framework that explicitly incorporates document context by jointly encoding multiple pages and deriving page-level
embeddings from a shared contextual representation. Furthermore, we introduce CMCL, a contextual
multimodal contrastive learning objective that effectively trains CMDR-Embed by balancing contextual modeling with page-level discriminability. Experiments demonstrate that CMDR-Embed significantly outperforms non-contextual
embeddings, highlighting the importance of context-aware
multimodal embeddings for advancing document
retrieval.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrails
Mingyang Song, Luxin Xu, Haoyu Sun et al. (+3 more)
Image guardrails are typically trained and evaluated under a fixed safety policy, implicitly treating safety as an intrinsic property of an image. Real deployments are different: the same image may be allowed in one product, restricted in another, and newly disallowed when a policy boundary changes. We study policy-adaptive image guardrailing, where a model must decide whether an image violates the currently supplied policy and generalize to held-out policy definitions. We introduce PolicyShiftBench, a comprehensive
benchmark with 2,000 policy-discriminative instances over 265 images, where each image is paired with 7.55 policy-conditioned
prompts on average to test whether models adapt to the active policy rather than relying on image-level safety priors. We then propose PolicyShiftGuard, a compact policy-conditioned guardrail trained with a two-stage training recipe that combines Randomized Policy SFT (RP-SFT) with Boundary-Pair Policy Adaptation (BP-Adapt). BP-Adapt trains matched
prompts for the same image and risk category using standard label supervision and a pairwise comparison loss that separates blocking policies from passing policies. Experiments show that existing VLMs and specialized guardrails remain brittle under policy shifts, while PolicyShiftGuard substantially improves policy-sensitive performance. The 7B model achieves SOTA performance of 76.9 Avg. F1 and 72.1 Avg. PSS on PolicyShiftBench, transfers well to UnSafeBench and SafeEditBench, and improves the latency-performance trade-off with a concise output format. Ablations confirm that matched pass/block boundary pairs are essential for stable policy adaptation.
Read the full abstract ▾
All authors Mingyang Song, Luxin Xu, Haoyu Sun, Minzhou Pan, Yu Cheng, Bo Li
Computation and Language · Cross-listed from cs.LG
K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)
Jean-Francois Bonbhel
We present K-ABENA (K-Adaptive Backpropagation with Error-based N-exclusion
Algorithm), a selective gradient computation framework that reduces per-iteration training cost by excluding a fraction of low-loss ("minor") observations from the backward pass. Its canonical form (v3) combines a defensive-mixture sampling design over the minor set with Horvitz-Thompson inverse-probability reweighting, yielding a design-unbiased Horvitz-Thompson gradient
estimator (Lemma 2) and whose self-normalized practical variant carries a bias of order O(1/m) with an explicit constant (Lemma 3). We prove an O(1/sqrt(T)) non-convex convergence guarantee for SGD under the
estimator, with an additive term that quantifies the residual bias (
Theorem 1). We further prove that uncompensated loss-based selection - a family that includes OHEM, SBP, and the two earlier K-ABENA variants - admits no stationary point at any minimizer where its selection bias is bounded away from zero (Proposition 2), and we quantify this failure empirically: at 0.17% class imbalance, uncompensated variants reach test AUC 0.53-0.62 versus 0.9998 for full-batch SGD, while the compensated
estimator attains 0.9991 at identical 28.4% compute savings. On real
datasets (Breast Cancer, Digits, Wine, Diabetes) the compensated
estimator is statistically indistinguishable from full-batch SGD (paired permutation tests, p >= 0.5; Section 7) while saving 28-54% of per-epoch gradient computation. A biased "regularized mode" (the earlier half-domain variant) is retained as an option with a proven exact bias decomposition (Lemma 5) and quantified contraindications: it collapses to 0.386 accuracy under 40% label noise (baseline: 0.832) and to 0.53 AUC under extreme imbalance. Every advantage and every limitation reported in this paper is either proved or measured; all experiments are CPU-scale (NumPy/scikit-learn) and their scope is stated explicitly.
Read the full abstract ▾
Computation and Language
Mitigating Factual Hallucination in Large Reasoning Models via Mixed-Mode Advantage Regularization
Kaishen Wang, Tong Zheng, Xuehao Cui et al. (+3 more)
Large
reasoning models (LRMs) improve
language model capabilities by generating explicit thinking traces before final answers. In factuality-oriented question answering (QA), such thinking often improves overall performance by helping the model recover relevant knowledge and refine its answers. However, we find that this benefit is not uniform at the instance level: explicit thinking can also overturn correct non-thinking answers and lead to factual drift. We refer to this failure mode as thinking-induced
hallucination. To explain this phenomenon, we formulate explicit thinking in factuality QA as a thinking residual over the model's direct-answer tendency, which can either recover missing knowledge or introduce unsupported associations. Based on this formulation, we propose MARGO, \underline{M}ixed-Mode \underline{A}dvantage \underline{R}egularization for \underline{G}rounded \underline{O}ptimization, a reinforcement learning framework that uses non-thinking rollouts as same-model references in advantage estimation. By constructing mixed-mode rollout groups with both thinking and non-thinking trajectories, MARGO evaluates whether explicit thinking adds factual value beyond direct answering, thereby suppressing
hallucination-prone thinking while preserving beneficial thinking behaviors. Experiments across multiple factuality-oriented QA
benchmarks demonstrate that MARGO improves factual reliability over strong baselines, while evaluations on mathematical
benchmarks show that it preserves general
reasoning ability.
Read the full abstract ▾
All authors Kaishen Wang, Tong Zheng, Xuehao Cui, Ruibo Chen, Tianyi Xiong, Heng Huang
Computation and Language
CoPiT: Cognitive Pivot Translation for Digraphic Low-Resource Mongolian in the Traditional Script
Burte Bayarsaikhan, Serynn Kim and Buru Chang
Low-resource
languages remain challenging for machine translation, and Mongolian is a representative case. As a digraphic
language, Mongolian is written in both Cyrillic and Traditional scripts, which exhibit a severe imbalance in data availability. While the Cyrillic script is relatively well-resourced, the Traditional script remains extremely data-scarce and orthographically ambiguous, leading to substantial performance degradation in direct translation. We propose CoPiT, a cognitively motivated pivot-based translation pipeline that exploits this internal resource hierarchy by routing translation through the Cyrillic script. The pipeline explicitly resolves script-induced ambiguity in the Traditional script before translation, enabling more stable and accurate meaning transfer. Across multiple backbone models and target
languages, CoPiT consistently outperforms direct translation, achieving substantial absolute BLEU improvements together with consistent 1.5-1.6x COMET gains. These gains allow strong open-source models to match or outperform GPT-4.1 under comparable evaluation settings. Beyond inference-time improvements, CoPiT enables the construction of synthetic parallel data directly from Traditional-script text, mitigating data scarcity in realistic low-resource scenarios. We release a new multi-script parallel
dataset covering Mongolian in both scripts alongside English, Korean, and Russian. All
datasets and code are publicly available at https://anonymous.4open.science/r/anonymous_project-76C7.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
StateFuse: Deterministic Conflict-Preserving Memory for Multi-Agent Systems
Sergey Volkov, Yang Li and Ye Luo
Agent systems accumulate conflicting observations across branches, retries, and replicas, yet many practical
memory layers still collapse disagreement behind overwrite rules that are difficult to inspect or correct. We present StateFuse, a conflict-aware replicated
memory contract built on standard OpSet/CRDT merge. StateFuse does not introduce a new join
algebra; it defines an
agent-facing semantics layer with immutable history, explicit conflict objects, exact and semantic correction handles (claim_id / claim_ref), deterministic predicate contracts, and projection-time resolution that cannot rewrite replicated state. We evaluate StateFuse against flat multi-value, raw-log, provenance-style, and collapsed baselines under matched resolver and verification policies. On a 282-question official conflict-bearing MemoryAgentBench slice, the compared methods tie on answer accuracy, but conflict-preserving surfaces keep contradictions visible while collapsed surfaces do not. In a controlled
agent loop with uniform verification, preserving ambiguity enables safer abstention and correction than early collapse. A correction-handle ablation further shows that semantic handles matter when exact prior identifiers are unavailable. The resulting claim is narrow: StateFuse is best supported as a safer public
memory contract for contradiction surfacing, abstention, and auditable correction, not as a universal accuracy gain.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
TurnOPD: Making On-Policy Distillation Turn-Aware for Efficient Long-Horizon Agent Training
Yuhang Zhou, Kai Zheng, Haoling Li et al. (+3 more)
On-policy
distillation (OPD) trains a student policy by matching a stronger teacher on the student's own trajectories, offering a promising framework for
language agent training. However, its application to long-horizon agentic tasks remains insufficiently explored. We identify two key inefficiencies in vanilla
agent OPD: (1) full-horizon rollouts often waste wall-clock resources on tail turns that provide weak and noisy KL supervision, and (2) trajectory-level KL objectives concentrate most of the loss on shallow
tokens, leaving deeper decision turns under-trained once initial behaviors are aligned. To address these challenges, we propose TurnOPD, a turn-level budgeting strategy for efficient on-policy
distillation of long-horizon
agents. TurnOPD consists of two budget controllers: adaptive rollout-depth budgeting, which uses probe-based turn statistics to determine rollout length, and progressive turn-normalized loss budgeting, which gradually shifts KL weighting from
token-level to turn-balanced supervision. Experiments on ALFWorld, WebShop, and Multi-Hop Search with task-specialized teacher models show that TurnOPD achieves superior validation accuracy under equal wall-clock training budgets and advances the accuracy–time frontier beyond vanilla OPD.
Read the full abstract ▾
All authors Yuhang Zhou, Kai Zheng, Haoling Li, Dengyun Peng, Can Xu, Jingjing Chen
Computation and Language
Inject or Navigate? Token-Efficient Retrieval for LLM Analysis of Transactional Legal Documents
Mahmoud Hany, Mourad ElSheraey, Mahmoud Said et al. (+1 more)
Answering questions over a set of transactional legal documents is most simply done by injecting the whole corpus into the LLM's context window on every query. That baseline maximises
retrieval recall, but its
token footprint scales with the corpus rather than the question, and long-context degradation scales with it. We report what it took to replace full-corpus injection in a legal-document analysis system, comparing it against two structured
retrieval modes over our proprietary structure-aware chunking:
embedding retrieval (NAVEMBED) and LLM
navigation over a compact structured index (NAVINDEX). On a 20-question
benchmark with verified ground-truth answers, a position-bias-controlled, reference-anchored pairwise judge scored semantic
retrieval with reranking tied with injection on 16 of 18 document-bound questions (injection preferred on 2) while attending to 17.3x fewer input
tokens (a general-text-
embedding (GTE) configuration reaches 29.9x at a lower tie rate); both modes were judged tied on the 2 out-of-scope controls. NAVINDEX was judged tied on all 18 at a 1.61x smaller total
token footprint, a ~56x smaller answering context, and 25% lower dollar cost. We derive a closed-form caching-crossover rule: cached injection is cheaper in dollars only while the corpus stays below roughly ten times the
retrieval payload. Scope and uncertainty are quantified in Section 8.
Read the full abstract ▾
All authors Mahmoud Hany, Mourad ElSheraey, Mahmoud Said, Peter Naoum
Computation and Language
When Should LLMs Search? Counterfactual Supervision for Search Routing
Minho Kim
Search-augmented
language models can use external evidence to compensate for limitations in parametric knowledge, but search is not uniformly beneficial: models may call search for questions they can already answer, or rely on noisy evidence when correction, clarification, or abstention would be more appropriate. We formulate this as an instance-level search-routing problem: deciding whether search is needed to improve task success relative to a no-search execution. To derive supervision, we compare no-search and forced-search outcomes for the same question and construct an oracle over NO SEARCH, SEARCH, and UNSOLVED based on task-specific success. Using this oracle as both an evaluation criterion and a learning signal, we train search-routing policies with supervised fine-tuning and preference optimization, improving routing macro-F1 on oracle-eligible examples from 0.7082 to 0.8235 for Gemma E2B and from 0.7053 to 0.8365 for Qwen3.5-4B. Further analysis shows that the learned policies reduce model-specific routing failures: Gemma primarily learns no-search restraint, while Qwen further reduces missed search; residual UNSOLVED cases reveal heterogeneous bottlenecks involving model capacity,
retrieval budget, evidence use, and policy behavior.
Read the full abstract ▾