Computation and Language · Cross-listed from cs.LG
A Measure-Theoretic Analysis of Reasoning in Transformers: Structural Generalization and Approximation Limits
Yuyang Zhang, Yifu Zhang, Xuehai Zhou et al. (+1 more)
Scaling laws for
reasoning in large
language models (LLMs) are well documented, but little theory explains their out-of-distribution (OOD) generalization. We formalize
reasoning with optimal transport: each search trajectory is mapped to twelve structural coordinates, such as tree depth and backtracking rate, and
dataset shift is measured by the Wasserstein-1 distance. Kantorovich duality bounds the OOD risk through this distance, the architecture's Lipschitz constant and approximation limits. First, under absolute positional encoding (APE), shifting part of a trajectory to later positions almost surely changes the
attention scores within that part; if this misroutes exact-match search, the Lipschitz constant and risk are $\Omega(1)$. Shift-invariant encodings such as rotary position
embedding (RoPE) leave the scores unchanged and bound the error. Second, we map backtracking to a Dyck-$k$
language and, under an implicit-depth assumption, derive a risk floor for search trees deeper than a multiple of the number of layers. Enough layers remove this floor; more width only shrinks a separate Barron-space approximation term. Experiments with 54 trained
Transformers on combinatorial search (Countdown) support these bounds. Appendix experiments measure structural shifts in public
agent,
embodied and
robot trajectories, shift all position indices of 33 public
language models, and split deeply nested tasks into separate model calls.
Read the full abstract ▾
All authors Yuyang Zhang, Yifu Zhang, Xuehai Zhou, Xiaoyin Chen
Computation and Language
What Are LLMs Doing to Scientific Communication? Measuring Changes in Writing Practices and Reading Experience
Filip Miletić and Neele Falk
Has the style of scientific communication changed due to the growing use of large
language models in the writing process? We address this question in the domain of Natural
Language Processing by leveraging two data resources we create: a naturalistic corpus of over 37,000 papers from the ACL Anthology (2020-2024); and a synthetic
dataset of 3,000 human-written passages and their LLM-generated improvements. We first implement a series of diachronic lexical analyses, showing that both word frequency and usage contexts have changed significantly over time, indicating semantic specialization in some cases and generalization in others. Broadening our perspective, we then model a range of more complex stylistic features and find that LLM-modified texts more frequently contain certain syntactic constructions, more complex and longer words and a lower lexical diversity. Finally, we connect these changes in writing practices to subjective reading experience through a pilot annotation study with 20 domain experts. They overall rate LLM-improved texts as more understandable and exciting, but also express negative qualitative attitudes towards LLMs, highlighting the strongly subjective effect of AI-assisted writing on reading experience.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents
Zhuohan Gu, Qizheng Zhang, Omar Khattab et al. (+1 more)
Large
language model (LLM)
agents increasingly operate over long and recurring external contexts, like document corpora and code repositories. Across invocations, existing approaches preserve either the
agent's trajectory, passive access to raw material, or task-level strategies. None of them preserves what we argue is most needed for repeated same-context workloads: reusable orientation knowledge (e.g., what the context contains, how it is organized, and which entities, constants, and schemas have historically been useful) about the recurring context itself. We introduce PEEK, a system that caches and maintains this orientation knowledge as a context map: a small, constant-sized artifact in the
agent's
prompt that gives it a persistent peek into the external context. The map is maintained by a programmable cache policy with three modules: a Distiller that extracts transferable knowledge from inference-time signals, a Cartographer that translates it into structured edits, and a priority-based Evictor that enforces a fixed
token budget. On long-context
reasoning and information aggregation, PEEK improves over strong baselines by 6.3-34.0% while using 93-145 fewer iterations and incurring 1.7-5.8x lower cost than the state-of-the-art
prompt-learning framework, ACE. On context learning, PEEK improves solving rate and rubric accuracy by 6.0-14.0% and 7.8-12.1%, respectively, at 1.4x lower cost than ACE. These gains generalize across LMs and
agent architectures, including OpenAI Codex, a production-grade coding
agent. Together, these results show that a context map helps long-context LLM
agents interact with recurring external contexts more accurately and efficiently.
Read the full abstract ▾
All authors Zhuohan Gu, Qizheng Zhang, Omar Khattab, Samuel Madden
Computation and Language
Where Does Authorship Signal Emerge in Encoder-Based Language Models?
Francis Kulumba, Guillaume Vimont, Laurent Romary et al. (+1 more)
Authorship attribution models fine-tuned with the same pretrained encoder, data, and loss can differ four-fold in performance depending only on their scoring mechanism. We use mechanistic interpretability tools to explain this gap. Stylistic features such as word length, punctuation density, and function-word frequency are similarly available at every layer in every model we probe, including an off-the-shelf control encoder, suggesting that the gap is not explained by their linear readability. Instead, causal intervention shows that the scorer appears to determine where the encoder consolidates authorship signal. Mean pooling forces consolidation by early to mid layers, while late interaction defers it to later layers. We further derive this difference from the gradient structure of each scorer, and training dynamics reveal distinct learning trajectories that follow from that difference.
Read the full abstract ▾
All authors Francis Kulumba, Guillaume Vimont, Laurent Romary, Florian Cafiero
Computation and Language
Are Tools Always Beneficial? Learning to Invoke Tools Adaptively for Dual-Mode Multimodal LLM Reasoning
Qinghe Ma, Zhen Zhao, Yiming Wu et al. (+3 more)
Tool-augmented
reasoning has emerged as a promising direction for enhancing the
reasoning capabilities of
multimodal large
language models (MLLMs). However, existing studies mainly focus on enabling models to perform tool invocation, while neglecting the necessity of invoking tools. We argue that tool usage is not always beneficial, as redundant or inappropriate invocations largely increase
reasoning overhead and even mislead model predictions. To address this issue, we introduce AutoTool, a model that adaptively decides whether to invoke tools according to the characteristics of each query. Within a reinforcement learning framework, we design an explicit dual-mode
reasoning strategy with mode-specific reward functions to guide the model toward producing accurate responses. Moreover, to prevent premature bias toward a single
reasoning mode, AutoTool jointly explores and balances tool-assisted and text-centric
reasoning throughout training, and promotes free exploration in later stages. Extensive experiments demonstrate that AutoTool exhibits outstanding performance and high efficiency, yielding a 21.8% accuracy gain on V*
benchmark compared to the base model, and a 44.9% improvement in efficiency over existing tool-augmented methods on POPE
benchmark. Code is available at https://github.com/MQinghe/AutoTool.
Read the full abstract ▾
All authors Qinghe Ma, Zhen Zhao, Yiming Wu, Jian Zhang, Lei Bai, Yinghuan Shi
Computation and Language
CLIF: Concept-Level Influence Functions for Transparent Bottleneck Models
Yike Sun, Mingkun Xu, Mu You et al. (+5 more)
In recent years, the black-box nature of deep learning models has limited their application in high-stakes domains such as medical diagnosis and finance, where interpretability is essential. To address this, we propose a novel approach using influence functions to enhance interpretability in NLP models at both the sample and concept levels. Experiments on CEBaB and Yelp
datasets show that influence functions effectively identify the most impactful training samples, both helpful and harmful, on model predictions. By adjusting the labels and weights of these samples, we demonstrate that model performance can be restored to baseline levels without retraining, confirming the value of influence functions for efficient data debugging. Furthermore, our concept-level analysis identifies key concepts within Concept Bottleneck Models (CBM) that significantly affect predictions. Modifying these concepts alters model behavior observably, providing clear insights into the decision process.
Read the full abstract ▾
All authors Yike Sun, Mingkun Xu, Mu You, Zhongzhi He, Henghua Shen, Zehan Tan, Derek F. Wong, Tao Fang
Computation and Language · Cross-listed from cs.CV
FineBench: Benchmarking and Enhancing Vision-Language Models for Fine-grained Human Activity Understanding
Gueter Josmy Faure, Min-Hung Chen, Jia-Fong Yeh et al. (+2 more)
Vision-
Language Models (VLMs) have demonstrated remarkable capabilities in general
video understanding, yet they often struggle with the fine-grained comprehension crucial for real-world applications requiring nuanced interpretation of human actions and interactions. While some recent human-centric
benchmarks evaluate aspects of model behaviour such as
fairness/ethics,
emotion perception, and broader human-centric metrics, they do not combine long-form
videos, very dense QA coverage, and frame-level spatial/temporal grounding at scale. To bridge this gap, we introduce FineBench, a human-centric
video question answering (VQA)
benchmark specifically designed to assess fine-grained understanding. FineBench comprises 199,420 multiple-choice QA pairs densely annotated across 64 long-form
videos (15 minutes each), focusing on detailed person movement, person interaction, and object manipulation, including compositional actions. Our extensive evaluation reveals that while proprietary models like GPT-5 achieve respectable performance, current open-source VLMs significantly underperform, struggling particularly with spatial
reasoning in multi-person scenes and distinguishing subtle differences in human movements and interactions. To address these identified weaknesses, we propose FineAgent, a modular framework that enhances VLMs by leveraging a Localizer and a Descriptor. Experiments show that FineAgent consistently improves the performance of various open VLMs on FineBench. FineBench provides a rigorous testbed for future research into fine-grained human-centric
video understanding, while FineAgent offers a practical approach to enhance such
reasoning in current VLMs. Project page and code at https://joslefaure.github.io/assets/html/finebench.html.
Read the full abstract ▾
All authors Gueter Josmy Faure, Min-Hung Chen, Jia-Fong Yeh, Hung-Ting Su, Winston H. Hsu
Computation and Language · Cross-listed from cs.CV
CADENet: Condition-Adaptive Asynchronous Dual-Stream Enhancement Network for Adverse Weather Perception in Autonomous Driving
Sherif Khairy and Catherine M. Elias
Adverse weather (rain, fog, sand, and snow) degrades camera-based object detection in autonomous vehicles. Existing enhancement-then-detect approaches stall the safety-critical perception loop, violating hard real-time requirements. Progress on this problem is also constrained by an under-recognized evaluation ceiling: ground truth annotated on degraded images cannot credit a detector that recovers objects the annotators themselves could not see, so a genuinely useful enhancement can register as a near-flat F1 gain. This paper presents CADENet (Condition-Adaptive Asynchronous Dual-stream Enhancement Network), a training-free three-thread system: Thread S (YOLOv11n) delivers detections at full frame rate with zero added latency; Thread Q applies condition-adaptive enhancement (CAPE) and fuses results via
entropy-guided NMS (EG-NMS) without blocking Thread S; Thread E provides CLIP zero-shot weather classification, so new weather categories require only a new text
prompt, with no labeled data and no retraining. Evaluated on 1327 DAWN images (YOLOv11m, IoU = 0.5, confidence = 0.25), CADENet achieves Recall = 0.0103 (micro), F1 = 0.0230 on snow, and F1 = 0.0038 on rain. We formalize the annotation completeness bias on DAWN-class data, so the reported F1 values are lower bounds on the true gain; recall is the annotation-gap-immune headline metric. Thread S sustains approximately 44 FPS regardless of enhancement load. No model retraining or additional sensor hardware is required.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation
Zhifei Xie, Kaiyu Pang, Haobin Zhang et al. (+4 more)
Despite rapid advances in automatic
speech recognition (ASR) and large audio-
language models, robust recognition in real-world environments remains limited by an "acoustic robustness bottleneck": models often lose acoustic grounding and produce omissions or
hallucinations under severe, compositional distortions. We propose Mega-ASR, a unified ASR-in-the-wild framework that combines scalable compound-data construction with progressive acoustic-to-semantic optimization. We introduce Voices-in-the-Wild-2M, covering 7 classic acoustic phenomena and 54 physically plausible compound scenarios, and train Mega-ASR with Acoustic-to-Semantic Progressive Supervised Fine-Tuning and Dual-Granularity WER-Gated Policy Optimization. Extensive experiments demonstrate that Mega-ASR achieves significant advantages over prior state-of-the-art systems on adverse-condition ASR
benchmarks (45.69% vs. 54.01% on VOiCES R4-B-F, and 21.49% vs. 29.34% on NOIZEUS Sta-0). On complex compositional acoustic scenarios, Mega-ASR further delivers over 30% relative WER reduction against strong open- and closed-source baselines, establishing a scalable paradigm for robust ASR in-the-wild.
Read the full abstract ▾
All authors Zhifei Xie, Kaiyu Pang, Haobin Zhang, Deheng Ye, Xiaobin Hu, Shuicheng Yan, Chunyan Miao
Computation and Language · Cross-listed from cs.AI
From Prompts to Pavement Through Time: Temporal Grounding in Agentic Scene-to-Plan Reasoning
Ahmed Y. Gado, Omar Y. Goba, Alaa Hassanein et al. (+2 more)
Recent attempts to support high-level scene interpretation and planning in Autonomous Vehicles (AVs) using ensembles of Large
Language Models (LLMs) and Large
Multimodal Models (LMMs) continue to treat time as a secondary property. This lack of temporal grounding leads to inconsistencies in
reasoning about continuous actions, undermining both safety and interpretability. This work explores whether temporal conditioning within inter-
agent communication can preserve or enhance coherence without introducing degradation in semantic or logical consistency. To investigate this, we introduce three planner architectures with progressively increasing temporal integration and evaluate them on curated subsets of the BDD-X
dataset using semantic, syntactic, and logical metrics. Results show that while temporal conditioning reshapes
reasoning style, it yields no statistically significant improvements in standard NLP-based correctness metrics. However, qualitative analysis reveals predictive hazard
reasoning, stable corrective behavior, and strategic divergence in the Sentinel. These findings clarify the limits of
prompt-based temporal grounding and establish the first empirical
benchmark for temporal scene-to-plan
reasoning.
Read the full abstract ▾
All authors Ahmed Y. Gado, Omar Y. Goba, Alaa Hassanein, Catherine M. Elias, Ahmed Hussein
Computation and Language
LP-Eval: Rubric and Dataset for Measuring the Quality of Legal Proposition Generation
Shanshan Xu, Johan Lindholm, Amogh Raina et al. (+2 more)
Legal proposition generation is central to legal
reasoning and doctrinal scholarship, yet remain under-examined in Legal NLP. This paper investigates the automatic generation and evaluation of legal propositions from decisions of the Court of Justice of the European Union using large
language models (LLMs). We introduce LP-Eval, a three-step evaluation rubric co-designed with legal experts that decomposes legal proposition quality into formal validity and substantive dimensions. Using this rubric, we release a
dataset of two experts' annotations for 100 LLM-generated legal propositions. Our results show that LLMs can generate predominantly well-formed and high-quality propositions, while expert evaluations reveal higher quality for propositions derived from well established cases than from recent ones. We further examine LLMs as evaluators and find that rubric-guided LLM judgments align more closely with expert assessments than direct overall scoring, but remain insensitive to finer-grained distinctions captured by human experts.
Read the full abstract ▾
All authors Shanshan Xu, Johan Lindholm, Amogh Raina, Henrik Palmer Olsen, Daniel Hershcovich
Computation and Language
Chunking German Legal Code
Max Prior, Natalia Milanova and Andreas Schultz
This paper investigates chunking strategies for
retrieval-augmented generation on German statutory law, using the German Civil Code as a structured
benchmark corpus. We implement and compare a range of
segmentation approaches, including structural units (sections, subsections, sentences, propositions), fixed-size windows, contextual chunking, semantic clustering, Lumber-style chunking, and RAPTOR-based hierarchical
retrieval. All methods are evaluated on a legal question-answering
dataset with section-level gold labels, measuring recall, query latency, index build time, and storage requirements. Results show that chunking strategies aligned with the inherent legal structure - particularly section and subsection - based
retrieval-achieve the highest recall, while more complex approaches that override this structure perform worse. These simpler methods also offer favorable computational efficiency compared to LLM-intensive techniques such as contextual chunking, RAPTOR, and Lumber. The findings highlight a key trade-off between semantic enrichment and operational cost, and demonstrate that preserving domain-specific structure is critical for effective legal information
retrieval.
Read the full abstract ▾
Computation and Language
Towards Trust Calibration in Socially Interactive Agents: Investigating Gendered Multimodal Behaviors Generation with LLMs
Lucie Galland, Chloé Clavel and Magalie Ochs
As Socially Interactive
Agents (SIAs) become increasingly integrated into daily life, the ability to calibrate user trust to an
agent's actual capabilities would help ensure appropriate usage of these
agents. In this paper, we explore the capacity of Large
Language Models (LLMs) to generate
multimodal behaviors (verbal, vocal, gestural, and facial expression modalities) that reflect varying levels of ability and benevolence, two key dimensions of trustworthiness. We propose a novel method for automatically generating behaviors aligned with specific levels of these traits, a first step towards enabling nuanced and trust-calibrated interactions. By analyzing a large
dataset of
multimodal transcripts generated by LLMs, we demonstrate that GPT-5.4 is able to produce coherent behavior across different modalities (text, intonation, facial expression, and gesture). Using Random Forest feature importance analysis, we show that the generated behaviors align with theoretical expectations for ability and benevolence. However, we also find that when gender is specified in the
prompt, LLMs tend to reproduce societal gender stereotypes, associating male
agents' behaviors with high ability and female
agents' behaviors with high benevolence. To validate our approach, we conducted a user study on Prolific using a within-subjects design. Participants perceived different levels of ability and benevolence in the generated behaviors align with the intended instructions.
Read the full abstract ▾
Computation and Language
Synthesis and Evaluation of Long-term History-aware Medical Dialogue
Hebin Hu, Renke Dai, Ah-Hwee Tan et al. (+1 more)
An effective healthcare
agent must be able to recall and reason over a patient's longitudinal medical history. However, the absence of
datasets with realistic long-term dialogue timelines limits systematic evaluation. Real clinical text is constrained by
privacy and ethics, while existing
benchmarks focus on isolated interactions, failing to capture cross-session
reasoning. We introduce a framework for synthesizing high-quality, long-term medical dialogues with LLMs. Our approach entails a knowledge-guided decomposition into three stages: constructing synthetic patient profiles with diverse disease and complication trajectories, generating multi-turn dialogues per encounter, and integrating them into a coherent longitudinal history
dataset, MediLongChat. We establish three
benchmark tasks-In-dialogue
Reasoning, Cross-dialogue
Reasoning, and Synthesis
Reasoning-to evaluate the
memory capabilities of healthcare
agents. To assess data quality, we introduce a multi-dimensional evaluation framework combining vector-based metrics with LLM-as-a-judge assessments. Specifically, we define automatic measures-Faithfulness, Coherence, and Diversity-together with two LLM-based evaluations: Correctness and Realism.
Benchmark experiments show that even state-of-the-art LLMs struggle with MediLongChat. These findings highlight the
benchmark's applicability and underscore the need for tailored methods to advance healthcare
agents.
Read the full abstract ▾
All authors Hebin Hu, Renke Dai, Ah-Hwee Tan, Yilin Kang
Computation and Language · Cross-listed from cs.AI
What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code
Yuze Zhao, Junpeng Fang, Lu Yu et al. (+6 more)
Code has become a standard component of modern foundation
language model (LM) training, yet its role beyond programming remains unclear. We revisit the claim that code improves
reasoning through controlled pretraining experiments on a 10T-
token corpus with fine-grained domain separation. Our findings are threefold. First, when code is restricted to standalone executable programs and Code-NL data are controlled for, code substantially improves programming ability but does not act as a general
reasoning enhancer; instead, it competes with knowledge-intensive tasks, especially complex mathematical
reasoning. Second, the
reasoning gains often attributed to code are better explained by cross-domain structured
reasoning traces, such as code-text and math-text mixtures, rather than by executable code alone. Third, increasing the density of structured math-domain samples within a fixed math budget yields substantial gains on difficult mathematical
reasoning while largely preserving programming performance, suggesting that cognitive scaffolds offer a targeted way to mitigate cross-domain trade-offs. Finally, routing analyses show that data-composition effects are reflected in expert-activation patterns, providing mechanism-level evidence for competitive and synergistic interactions across domains. Our results clarify which data characteristics transfer across capability dimensions and point to more precise data-centric optimization strategies.
Read the full abstract ▾
All authors Yuze Zhao, Junpeng Fang, Lu Yu, Zhenya Huang, Kai Zhang, Qing Cui, Qi Liu, Jun Zhou, Enhong Chen
Computation and Language
TERGAD: Structure-Aware Text-Enhanced Representations for Graph Anomaly Detection
Wen Shi, Zhe Wang, Huafei Huang et al. (+6 more)
Graph Anomaly Detection (GAD) aims to identify atypical graph entities, such as nodes, edges, or substructures, that deviate significantly from the majority. While existing text-rich approaches typically integrate structural context into the data representation pipeline using raw textual features, they often neglect the structural context of nodes. This limitation hinders their ability to detect sophisticated anomalies arising from inconsistencies between a node's inherent content and its topological role. To bridge this gap, we propose TERGAD (Structure-aware Text-enhanced Representations for Graph Anomaly Detection), A novel data augmentation framework that enriches structural semantics for GAD via the semantic
reasoning capabilities of Large
Language Models (LLMs). Specifically, TERGAD translates node-level topological properties into descriptive natural
language narratives, which are subsequently processed by an LLM to derive high-level semantic
embeddings. These
embeddings are then adaptively fused with original node attributes through a gated dual-branch autoencoder to jointly reconstruct both graph structure and node features. The anomaly score is computed based on the integrated reconstruction error, effectively capturing deviations in both observable attributes and LLM-informed semantic expectations. Extensive experiments on six real-world
datasets demonstrate that TERGAD consistently outperforms state-of-the-art baselines. Furthermore, our ablation studies validate the indispensable role of structural semantic guidance and the efficacy of the gated fusion mechanism. Code is available at https://github.com/Kantorakitty/TERGAD-main.
Read the full abstract ▾
All authors Wen Shi, Zhe Wang, Huafei Huang, Qing Qing, Ziqi Xu, Qixin Zhang, Xikun Zhang, Renqiang Luo, Feng Xia
Computation and Language
ContextRAG: Extraction-Free Hierarchical Graph Construction for Retrieval-Augmented Generation
Roman Prosvirnin, Sergei Kuznetsov and Seungmin Jin
Graph-structured
retrieval-augmented generation (RAG) systems can improve answer quality on multi-hop questions, but many current systems rely on large
language models (LLMs) to extract entities, relations, and summaries during indexing. These calls add
token and wall-clock costs that grow with corpus size. We present ContextRAG, a graph RAG system whose graph
topology is constructed without LLM-based entity or relation extraction. ContextRAG derives a fuzzy concept graph over chunk
embeddings using residual-quantization k-means and Formal Concept Analysis with Lukasiewicz residuated logic. Bridge-like and meet-derived context nodes are induced by soft fuzzy join and meet operations, rather than by LLM-written graph edges. On a 130-task UltraDomain subset, ContextRAG builds its index with 30 LLM calls and 22,073
tokens. In contrast, a local HiRAG reproduction stress test required 870 indexing calls and 3.54M
tokens on a 20-task subset before failing during graph construction; linear extrapolation to 130 tasks implies over 23M indexing
tokens. ContextRAG obtains 33.6% F1 overall and 36.8% F1 on multi-hop tasks. An activation analysis shows that queries retrieving at least one
lattice-derived node in the top five achieve +3.9 percentage points F1 over queries that do not; this association is diagnostic rather than causal.
Read the full abstract ▾
Computation and Language
Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
Husnain Amjad, Raja Khurram Shahzad, Aamir Shahzad et al. (+1 more)
Mathematical
reasoning is essential for problem-solving in education, science, and industry, serving as a crucial
benchmark for evaluating artificial intelligence systems. As Large
Language Models (LLMs) improve their
reasoning capabilities, understanding how well they perform mathematical
reasoning has become increasingly important. This survey synthesizes recent advancements in mathematical
reasoning with LLMs through a structured analysis of
datasets, architectures, training strategies, and evaluation protocols. Our systematic review encompasses approximately 120 peer-reviewed studies and preprints, examining the
evolution of this research area and providing a unified analytical framework to understand current progress and limitations. Our study particularly introduces a unified taxonomy of mathematical
datasets, distinguishing between pretraining corpora, supervised fine-tuning resources, and evaluation
benchmarks across varying levels of
reasoning complexity. A systematic analysis of
reasoning architectures and training strategies, including tool integration, verifier-guided
reasoning, and parameter-efficient adaptation, is presented to assess their effects on
reasoning robustness and generalization. Moreover, a comparative evaluation of existing metrics highlights the gap between final-answer accuracy and process-level
reasoning verification. By synthesizing insights across these areas, our analysis identifies recurring failure modes, such as
reasoning faithfulness issues,
benchmark biases, and generalization limitations, and outlines key research directions toward improving symbolic grounding, evaluation reliability, and the development of more robust and trustworthy LLM-based
reasoning systems.
Read the full abstract ▾
All authors Husnain Amjad, Raja Khurram Shahzad, Aamir Shahzad, Mehwish Fatima
Computation and Language
CAIT: A Syntactic Parsing Toolkit for Child-Adult InTeractions
Francesca Padovani, Xiulin Yang, Bastian Bunzeck et al. (+4 more)
CHILDES is a paramount resource for
language acquisition studies – yet computational tools for analyzing its syntactic structure remain limited. Leveraging the recent release of the UD-English-CHILDES treebank with gold-standard Universal Dependencies (UD) annotations, we train a state-of-the-art dependency parser specifically tailored to CHILDES. The parser more accurately captures syntactic patterns in child-adult interactions, outperforming widely used off-the-shelf English parsers, including SpaCy and Stanza. Alongside the parser, we also release a Part-of-
Speech tagger and an utterance-level construction tagger, which together form the open-source Syntactic Parsing Toolkit for Child-Adult InTeractions (CAIT). Through a detailed error analysis and a case study tracking the distribution of syntactic constructions across developmental time in CHILDES, we demonstrate the practical utility of the toolkit for large-scale, reproducible research on
language acquisition.
Read the full abstract ▾
All authors Francesca Padovani, Xiulin Yang, Bastian Bunzeck, Jaap Jumelet, Yevgen Matusevych, Nathan Schneider, Arianna Bisazza
Computation and Language
LLM-Based Financial Sentiment Analysis in Arabic: Evidence from Saudi Markets
Mona H. Albaqawi, Eman M. Albalkhi, Joud A. Albaiti et al. (+1 more)
Investor sentiment shapes financial
markets, yet modeling sentiment in Arabic financial contexts remains challenging due to linguistic complexity and limited resources. We present an Arabic NLP framework for large-scale financial sentiment analysis tailored to the Saudi
market, integrating official financial news and social media to capture institutional and public investor sentiment. The framework constructs a large Arabic financial corpus through a multi-stage pipeline encompassing data collection, cleaning, deduplication, entity linking, and sentiment annotation.
Transformer-based NER combined with a curated company lexicon links textual mentions to canonical company identifiers, with sentiment labels assigned using a five-class scheme. The resulting
dataset of 84K samples supports company-level sentiment aggregation and analysis of sentiment dynamics relative to stock
market behavior on the Saudi Exchange. Experimental results demonstrate reliable and scalable Arabic financial sentiment analysis.
Read the full abstract ▾
All authors Mona H. Albaqawi, Eman M. Albalkhi, Joud A. Albaiti, Enrico Lopedoto
Computation and Language
Can Large Language Models Reliably Correct Errors in Low-Resource ASR? A Contamination-Aware Case Study on West Frisian
Yun Hao, Reihaneh Amooie, Wietse de Vries et al. (+2 more)
Automatic
speech recognition (ASR) has improved substantially in recent years, yet performance remains limited for low-resource
languages. Large
language models (LLMs) have shown promise for improving ASR through generative error correction (GER), but their effectiveness in low-resource settings remains underexplored. In addition, it remains unclear to what extent data contamination influences the reported improvements in LLM-based GER. This study investigates LLM-based GER for low-resource Frisian. In addition to a public corpus, we construct and use a Frisian offline
dataset with non-public texts for evaluation to control for potential data contamination. Results show that GER improves ASR performance in most settings, with the best GPT-5.1 results surpassing oracle WERs. Comparable gains on the offline
dataset indicate that improvements reflect true correction ability. We further provide a detailed error analysis revealing model correction patterns.
Read the full abstract ▾
All authors Yun Hao, Reihaneh Amooie, Wietse de Vries, Rik van Noord, Martijn Wieling
Computation and Language · Cross-listed from cs.LG
OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
Zunhai Su, Rui Yang, Chao Zhang et al. (+12 more)
The rapid advancement toward long-context
reasoning and multi-modal intelligence has made the
memory footprint of the Key-Value (KV) cache a dominant
memory bottleneck for efficient deployment. While the established per-channel quantization effectively accommodates intrinsic channel-wise outliers in Key tensors, its efficacy diminishes under extreme compression. In this work, we revisit the inherent limitations of the per-channel quantization paradigm from both empirical and theoretical perspectives. Our analysis identifies
Token Norm Imbalance (TNI) as the primary bottleneck to quantization fidelity. We demonstrate that TNI systematically amplifies errors when shared quantization parameters are required to span
token groups exhibiting substantial norm disparities. Instead of relying on intricate quantization pipelines (e.g., TurboQuant), we propose OScaR (Omni-Scaled Canalized Rotation), an accurate and lightweight KV cache compression framework for X-LLMs (i.e., text-only, multi-modal, and omni-modal LLMs). Advancing the per-channel paradigm, OScaR employs Canalized Rotation followed by Omni-
Token Scaling to mitigate TNI-induced sequence-dimensional variance both effectively and efficiently, further supported by our optimized system design and CUDA kernels. Extensive evaluations across X-LLMs show that OScaR consistently outperforms existing methods and achieves near-lossless performance under INT2 quantization, establishing it as a robust, low-complexity, and universal framework that defines a new Pareto front. Compared with the BF16 FlashDecoding-v2 baseline, our OScaR implementation achieves a notable up to 3.0x speedup in decoding, reduces
memory footprint by 5.3x, and increases throughput by 4.1x. The code for OScaR is publicly available at https://github.com/ZunhaiSu/OScaR-KV-Quant.
Read the full abstract ▾
All authors Zunhai Su, Rui Yang, Chao Zhang, Yaxiu Liu, Yifan Zhang, Wei Wu, Jing Xiong, Dayou Du, Xialie Zhuang, Yulei Qian, Jianchao Tan, Yuchen Xie, Yik-Chung Wu, Hongxia Yang, Ngai Wong
Computation and Language
K-Quantization and its Impact on Output Performance
Robin Baki Davidsson and Pierre Nugues
Recent advancements in large
language models (LLMs) have shown their remarkable capacities in many NLP tasks. However, their substantial size often presents challenges for deployment. This necessitates efficient techniques for model compression, with quantization emerging as a prominent solution. Despite its benefits, the exact impact of quantization (from 2- to 6-bit) on the performance and accuracy of LLMs remains an active area of research. This paper investigates the performance of eight LLMs at various quantization levels, focusing on tasks such as MMLU-Pro for knowledge processing and
reasoning, CRUXEval for code comprehension, and MuSR for reading comprehension. Our results show a consistent trend where higher precision (e.g., 8-bit Q8\_0) yields improved performance, albeit with diminishing returns. Aggressive quantization (e.g., 2-bit Q2\_K) usually retains acceptable accuracy, though some models show a substantial loss in performance. Our findings indicate that while lower bit precision generally reduces performance, the impact varies across models and tasks. Larger models show greater resilience to aggressive quantization, but can still undergo significant drops at lower precision levels. Mid-sized models in the 7-9 billion parameter range strike an optimal balance between efficiency and resource usage. Such results provide insights into the trade-offs between model size, quantization, and performance.
Read the full abstract ▾
Computation and Language
optimize_anything: A Universal API for Optimizing any Text Parameter
Lakshya A Agrawal, Donghyun Lee, Shangyin Tan et al. (+11 more)
Can a single LLM-based optimization system match specialized tools across fundamentally different domains? We show that when optimization problems are formulated as improving a text artifact evaluated by a scoring function, a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs-achieves state-of-the-art results across six diverse tasks. Our system discovers
agent architectures that nearly triple Gemini Flash's ARC-AGI accuracy (32.5% to 89.5%), finds scheduling
algorithms that cut cloud costs by 40%, generates CUDA kernels where 87% match or beat PyTorch, and outperforms AlphaEvolve's reported circle packing solution (n=26). Ablations across three domains reveal that actionable side information yields faster convergence and substantially higher final scores than score-only feedback, and that multi-task search outperforms independent optimization given equivalent per-problem budget through cross-task transfer, with benefits scaling with the number of related tasks. Together, we show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm, unifying tasks traditionally requiring domain-specific
algorithms under a single framework. We open-source optimize\_anything with support for multiple backends as part of the GEPA project at https://github.com/gepa-ai/gepa .
Read the full abstract ▾
All authors Lakshya A Agrawal, Donghyun Lee, Shangyin Tan, Wenjie Ma, Karim Elmaaroufi, Rohit Sandadi, Sanjit A. Seshia, Koushik Sen, Dan Klein, Ion Stoica, Joseph E. Gonzalez, Omar Khattab, Alexandros G. Dimakis, Matei Zaharia
Computation and Language
HyperLogic: A Hard, Forward-Authored Chinese Logical Reasoning Benchmark with Execution-Derived Answers
Ming Zhang, Qiyuan Peng, Yinxi Wei et al. (+16 more)
Existing logic
benchmarks primarily measure models' ability to answer
reasoning questions directly. Scalable
benchmarks often generate text from formal structures, which makes answers easy to compute but fixes the formalization before the problem is written. Forward construction preserves the challenge of finding a faithful formalization, yet makes difficulty and answer reliability harder to control. We introduce HyperLogic, a forward-construction pipeline that separates problem authoring from answer generation. A multi-
agent workflow hardens undergraduate-authored Chinese seeds without solving them; two
agents from different model families independently translate each finished item into executable finite-domain models; their encodings and solver-derived answers undergo layered,
agent-assisted adjudication under human-expert oversight. HyperLogic-Base contains 195 items and 922 sub-questions and separates seven frontier models by 33.0 percentage points in strict item accuracy (44.6-77.6%). HyperLogic-Hard contains 100 items with larger, coupled search spaces, on which no model exceeds 16% accuracy in direct answering. We also use Hard to evaluate
agents' ability to formalize and solve problems with tools, comparing a code sandbox alone with one that includes our logic modeling library. The sandbox improves every model by 16.7-40.1 points; adding the library helps five models and hurts two. These results highlight the difficulty of faithful formalization even with tool access.
Read the full abstract ▾
All authors Ming Zhang, Qiyuan Peng, Yinxi Wei, Yujiong Shen, Kexin Tan, Yuhui Wang, Zhenghao Xiang, Junjie Ye, Zhangyue Yin, Zhiheng Xi, Shihan Dou, Weikang Wang, Yuhao Zhang, Tao Gui, Ruizhi Yang, Qi Zhang, Xuanjing Huang, Alex Chen, Maxm Pan
Computation and Language
GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
Minxuan Lv, Tiehua Mei, Tanlong Du et al. (+9 more)
We present GoLongRL, a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). Existing long-context RL methods often treat data construction as a matter of designing increasingly complex
retrieval paths, leading to homogeneous task coverage and reward formulations that inadequately reflect practical long-context requirements. Our work offers two contributions. (1) Capability-oriented data construction with full open release. We openly release a
dataset of 23K RLVR samples, the complete construction pipeline, and all training code. Guided by a taxonomy of long-context capabilities, the
dataset spans 9 task types, each paired with its natural evaluation metric. It comprises curated open-source samples from established corpora and synthetic samples whose QA pairs are generated from real source documents such as books, academic papers, and multi-turn dialogues. Under the same vanilla GRPO setup, our
dataset alone outperforms the closed-source QwenLong-L1.5
dataset. Moreover, our Qwen3-30B-A3B model trained on this data delivers long-context performance comparable to DeepSeek-R1-0528 and Qwen3-235B-A22B-Thinking-2507, suggesting that broader coverage and greater reward diversity substantially benefit long-context capability improvement. (2) TMN-Reweight for heterogeneous multitask optimization. To address optimization challenges from heterogeneous rewards, we propose TMN-Reweight, which combines task-level mean normalization for cross-task reward scale
alignment with difficulty-adaptive weighting for more reliable advantage estimation. TMN-Reweight further improves average performance over vanilla GRPO, with general capabilities preserved or improved across reported evaluations.
Read the full abstract ▾
All authors Minxuan Lv, Tiehua Mei, Tanlong Du, Junmin Chen, Zhenpeng Su, Ziyang Chen, Ziqi Wang, Zhennan Wu, Ruotong Pan, jian Liang, Ruiming Tang, Han Li
Computation and Language · Cross-listed from cs.AI
Library Drift: Diagnosing and Fixing a Silent Failure Mode in Self-Evolving LLM Skill Libraries
Xing Zhang, Yanwei Cui, Guanghui Wang et al. (+4 more)
Self-evolving skill libraries face a silent failure mode we term library drift: unbounded skill accumulation without outcome-driven lifecycle management causes
retrieval degradation, false-positive injections, and performance stagnation. Recent evaluation confirms the symptom (LLM-authored skills deliver +0.0pp gain while human-curated ones deliver +16.2pp (SkillsBench)), yet the underlying mechanism has not been isolated. We provide (1) a reproducible trigger: ablations that isolate drift: one disables skill injection (flat floor, +0.002), one imposes premature retirement (active harm, $-$0.019); (2) trace-level diagnostics: an append-only evidence log with per-skill contribution scores, attribution verdicts, and router engagement metrics that make the failure visible before it reaches end-task scores; and (3) a verified fix: a minimal governance recipe (outcome-driven retirement + bounded active-cap + meta-skill authoring prior) that lifts held-out pass@1 from a 0.258 baseline to a late-window mean of 0.584 (rolling gain $+$0.328) on MBPP+ hard-100 over 100 rounds. Eight ablations decompose which governance mechanisms are load-bearing and which are subsumed, providing a concrete playbook for diagnosing library drift in any self-evolving
agent.
Read the full abstract ▾
All authors Xing Zhang, Yanwei Cui, Guanghui Wang, Ziyuan Li, Wei Qiu, Bing Zhu, Peiyang He
Computation and Language
A Data-Driven Approach to Idiomaticity Based on Experts' Criteria in Theoretical Linguistics
Elena Mikhalkova, Anastasiya Vishnyakova, Anastasiya Drozdova et al. (+3 more)
The article observes data analysis of 286 multi-word expressions (MWEs) based on 16 lexical, grammatical and other criteria described in theoretical books and papers on the notion of idiomaticity. MWEs were collected from the same theoretical sources, and a set of experts in linguistics annotated them with these categories. The distribution of categories shows that there are no absolutely idiomatic expressions. Lexical criteria seem to be the most influential; grammatical criteria are bound to certain conditions; presence of obsolete words and grammar influence ability of an MWE to be replaced with one word.
Read the full abstract ▾
All authors Elena Mikhalkova, Anastasiya Vishnyakova, Anastasiya Drozdova, Polina Gavin, Aleksander Zhmykhov, Timofey Protasov
Computation and Language
m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder
Yaoxiang Wang, Simiao Zuo, Qingguo Hu et al. (+4 more)
Embedding models are pivotal in industrial information
retrieval systems like search and advertising. However, existing pretrained models often exhibit fixed architectures and
embedding dimensionalities, posing significant challenges when adapting them to diverse deployment scenarios with varying business-driven constraints. A common practice involves fine-tuning with partial parameter initialization from larger pretrained models for resource-constrained tasks. This method is often suboptimal as the misalignment between pretraining and downstream usage prevents full realization of pretraining benefits. To address this limitation, we introduce m3BERT: a Modern, Multi-lingual, Matryoshka Bidirectional Encoder, which features a novel pretraining strategy that jointly optimizes representations across both
transformer layers and multiple
embedding dimensions. This enables a single model to be tailored to varied resource and accuracy targets while maintaining consistency with pretraining. Incorporating recent architectural improvements, m3BERT uses a three-stage pretraining: monolingual pretraining,
multilingual adaptation to serve diverse user bases, and crucial continual pretraining on a massive web domain corpus to enhance utility in commercial
retrieval. m3BERT significantly outperforms state-of-the-art
embedding models in Bing-Click, a large-scale industrial
retrieval dataset, showcasing its practical versatility as an efficient foundation for resource-aware industrial
retrieval systems. Further experiments on public
datasets also confirm the general effectiveness of our multigranular Matryoshka pretraining strategy.
Read the full abstract ▾
All authors Yaoxiang Wang, Simiao Zuo, Qingguo Hu, Yucheng Ding, Yeyun Gong, Jian Jiao, Jinsong Su
Computation and Language
Investigating Cross-Modal Skill Injection: Scenarios, Methods, and Hyperparameters
Zhiyu Xu, Lean Wang, Yuanxin Liu et al. (+5 more)
Vision-
Language Models (VLMs) have demonstrated remarkable proficiency in general multi-modal understanding; yet they struggle to efficiently acquire continually evolving domain-specific skills. Conventional approaches to enhancing VLM capabilities, such as Supervised Fine-Tuning (SFT), require extensive
dataset curation and substantial computational resources. Model merging has emerged as an efficient alternative that enables the transfer of domain-specific expertise from Large
Language Models (LLMs) to VLMs without incurring additional training data requirements or significant computational overhead. Unlike conventional merging of homogeneous LLMs, which mainly aggregates existing capabilities, cross-modal skill injection aims to induce emergent cross-modal capabilities by integrating a domain-expert LLM into a VLM. However, existing research lacks a systematic analysis of the applicability and methodology of cross-modal skill injection. In this study, we investigate cross-modal skill injection across three main aspects: scenarios, methods, and hyperparameters. For scenarios, we find that cross-modal skill injection generally performs well in instruction-following and cross-lingual settings, yet struggles with mathematical
reasoning. For methods, we find that classic approaches such as TA and DARE consistently achieve superior performance over alternative merging methods. We also provide a systematic and quantitative analysis of the hyperparameter tuning that these classic methods critically depend on.
Read the full abstract ▾
All authors Zhiyu Xu, Lean Wang, Yuanxin Liu, Lei Li, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun