Computation and Language
Linear Semantic Segmentation for Low-Resource Spoken Dialects
Kirill Chirkunov, Younes Samih, Abed Alhakim Freihat et al. (+1 more)
Semantic
segmentation is a core component of discourse analysis, yet existing models are primarily developed and evaluated on high-resource written text, limiting their effectiveness on low-resource spoken varieties. In particular, dialectal Arabic exhibits informal syntax, code-switching, and weakly marked discourse structure that challenge standard
segmentation approaches. In this paper, we introduce a new multi-genre
benchmark (more than 1000 samples) for semantic
segmentation in conversational Arabic, focusing on dialectal discourse. The
benchmark covers transcribed casual telephone conversations, code-switched podcasts, broadcast news, and expressive dialogue from novels, and was annotated and validated by native Arabic annotators. Using this
benchmark, we show that
segmentation models performing well on MSA news genres degrade on dialectal transcribed
speech. We further propose a
segmentation model that targets local semantic coherence and robustness to discourse discontinuities, consistently outperforming strong baselines on dialectal non-news genres. The
benchmark and approach generalize to other low-resource spoken
languages.
Read the full abstract ▾
All authors Kirill Chirkunov, Younes Samih, Abed Alhakim Freihat, Hanan Aldarmaki
Computation and Language
Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learning
Ömer Faruk Akgül, Rajgopal Kannan, Willie Neiswanger et al. (+1 more)
Reinforcement learning has become the standard for improving
reasoning in large
language models, yet evidence increasingly suggests that RL does not teach new strategies; it redistributes probability mass over solutions the base model already contains. In this work, we ask: if RL merely steers the model toward paths it already knows, is the RL optimization loop itself necessary? Through
token-level analysis across multiple model families and RL
algorithms, we find that RL's beneficial footprint is a sparse, predictable correction concentrated at high-
entropy decision points where the model is uncertain which branch to take. Only 1–3% of
token positions are affected, the promoted
token always lies within the base model's top-5 alternatives, and targeted corrections at those few positions causally recover a large fraction of RL's accuracy gain, while random corrections fail. The base model's own
entropy identifies these positions without any RL-trained model, and the entire correction is low-dimensional, representable in a tiny fraction of model parameters. These findings reframe
reasoning improvement as sparse policy selection, not capability acquisition. We translate this insight into ReasonMaxxer, a minimal RL-free method that applies contrastive loss only at
entropy-gated decision points, using a few hundred base-model rollouts and no online generation. Across three model families, six scales, and six math
reasoning benchmarks, ReasonMaxxer matches or exceeds full RL performance while requiring only tens of problems and minutes of single-GPU training, a reduction in training cost of roughly three orders of magnitude.
Read the full abstract ▾
All authors Ömer Faruk Akgül, Rajgopal Kannan, Willie Neiswanger, Viktor Prasanna
Computation and Language
YEZE at SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization via Heterogeneous Ensembling
Fengze Guo and Yue Chang
This paper presents our system for SemEval-2026 Task 9: Detecting
Multilingual, Multicultural and Multievent Online Polarization, which identifies polarized social media content in 22
languages through three subtasks: binary detection, target classification, and manifestation identification. We propose a heterogeneous ensemble of
multilingual pretrained models, combining XLM-RoBERTa-large and mDeBERTa-v3-base. We investigate techniques such as multi-task learning, translation-based data augmentation, and class weighting to improve classification performance under severe label imbalance. Our findings indicate that independent task modeling combined with class weighting is more effective.
Read the full abstract ▾
Computation and Language
UniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic Sparsification
Qihang Fan, Huaibo Huang, Zhiying Wu et al. (+2 more)
As large
language models (LLMs) continue to advance rapidly, they are becoming increasingly capable while simultaneously demanding ever-longer context lengths. To improve the inference efficiency of long-context processing, several novel low-complexity hybrid architectures have recently been proposed, effectively alleviating the computational burden of long-context inference. However, existing research on long-context prefill acceleration remains predominantly focused on sparse
attention mechanisms, which achieve their maximum speedup only on full-
attention models. When transferred to emerging architectures–such as linear/full
attention hybrids or sliding window/full
attention hybrids–these prefill acceleration approaches suffer significant performance degradation. Furthermore, such methods are generally incompatible with continuous batching, making them difficult to integrate into modern inference engines such as vLLM. To this end, we propose UniPrefill, a prefill acceleration framework applicable to virtually any model architecture, which directly accelerates the model's computation at the
token level. We further implement UniPrefill as a continuous batching operator and extend vLLM's scheduling strategy to natively support prefill-decode co-processing and tensor parallel for UniPrefill, enabling its seamless integration into vLLM. UniPrefill achieves up to 2.1x speedup in Time-To-First-
Token (TTFT), with the acceleration becoming increasingly pronounced as the number of concurrent requests grows.
Read the full abstract ▾
All authors Qihang Fan, Huaibo Huang, Zhiying Wu, Bingning Wang, Ran He
Computation and Language
TIDE: Every Layer Knows the Token Beneath the Context
Ajay Jaiswal, Lauren Hannah, Han-Byul Kim et al. (+3 more)
We revisit a universally accepted but under-examined design choice in every modern LLM: a
token index is looked up once at the input
embedding layer and then permanently discarded. This single-injection assumption induces two structural failures: (i) the Rare
Token Problem, where a Zipf-type distribution of vocabulary causes rare-
token embeddings are chronically under-trained due to receiving a fraction of the cumulative gradient signal compared to common
tokens; and (ii) the Contextual Collapse Problem, where limited parameters models map distributionally similar
tokens to indistinguishable hidden states. As an attempt to address both, we propose TIDE, which augments the standard
transformer with EmbeddingMemory: an ensemble of K independent MemoryBlocks that map
token indices to context-free semantic vectors, computed once and injected into every layer through a depth-conditioned softmax router with a learnable null bank. We theoretically and empirically establish the benefits of TIDE in addressing the issues associated with single-
token identity injection as well as improve performance across multiple
language modeling and downstream tasks.
Read the full abstract ▾
All authors Ajay Jaiswal, Lauren Hannah, Han-Byul Kim, Duc Hoang, Mehrdad Farajtabar, Minsik Cho
Computation and Language · Cross-listed from cs.LG
Contrastive Identification and Generation in the Limit
Xiaoyu Li, Andi Han, Jiaojiao Jiang et al. (+1 more)
In the classical identification in the limit model of Gold [1967], a stream of positive examples is presented round by round, and the learner must eventually recover the target hypothesis. Recently, Kleinberg and Mullainathan [2024] introduced generation in the limit, where the learner instead must eventually output novel elements of the target's support. Both lines of work focus on positive-only or fully labeled data. Yet many natural supervision signals are inherently relational rather than singleton, which encode relationships between examples rather than labels of individual ones. We initiate the study of contrastive identification and generation in the limit, where the learner observes a contrastive presentation of data: a stream of unordered pairs $\{x,y\}$ satisfying $h(x)\ne h(y)$ for an unknown target binary hypothesis $h$, but which element is positive is hidden from the learner. We first present three results in the noiseless setting: an exact characterization of contrastive identifiable classes (a one-line
geometric refinement of Angluin [1980]'s tell-tale condition), a combinatorial dimension called contrastive closure dimension (a contrasitive analogue of the closure dimension in Raman et al. [2025]) and exactly characterizing uniform contrastive generation with tight sample complexity, and a strict hierarchy in which contrastive generation and text identification are mutually incomparable. We then prove a sharp reversal under finite
adversarial corruption: there exist classes identifiable from contrastive pairs under any finite corruption budget by a single budget-independent
algorithm, yet not identifiable from positive examples under even one corrupted observation. The unifying technical object is the common crossing graph, which encodes pairwise ambiguity, family-level generation obstructions, and corruption defects in a single coverage-and-incidence
language.
Read the full abstract ▾
All authors Xiaoyu Li, Andi Han, Jiaojiao Jiang, Junbin Gao
Computation and Language
A$^2$TGPO: Agentic Turn-Group Policy Optimization with Adaptive Turn-level Clipping
Dingwei Chen, Zefang Zong, Zhipeng Ma et al. (+5 more)
Reinforcement learning for agentic large
language models (LLMs) typically relies on a sparse, trajectory-level outcome reward, making it difficult to evaluate the contribution of individual tool-calls within multi-turn interactions. Existing approaches to such process credit assignment either depend on separate external process reward models that introduce additional consumption, or tree-based structural rollout that merely redistributes the outcome signal while constraining trajectory diversity. A promising alternative leverages the per-turn change in the policy's predicted probability of the ground-truth, termed Information Gain (IG), as an intrinsic process signal without an external evaluator. However, prior work on leveraging IG signals within the RL training loop faces three systematic challenges: normalizing across turns that face heterogeneous positional contexts can distort the relative standing of individual turns, accumulating a variable number of terms causes advantage magnitudes to drift with trajectory depth, and a fixed clipping range governs policy updates identically for turns with vastly different IG signals. In this paper, we propose A$^2$TGPO (Agentic Turn-Group Policy Optimization with Adaptive Turn-level Clipping), which retains IG as the intrinsic signal but re-designs how it is normalized, accumulated, and consumed: (i) turn-group normalization: normalizes IG within each (
prompt, turn-index) group so that each turn is compared only against peers at the same interaction depth; (ii) variance-rescaled discounted accumulation: divides cumulative normalized IG by square root of accumulated terms to keep advantage magnitudes comparable across turn positions; and (iii) adaptive turn-level clipping: modulates each turn's clipping range based on its normalized IG, widening the update region for informative turns and narrowing it for uninformative ones.
Read the full abstract ▾
All authors Dingwei Chen, Zefang Zong, Zhipeng Ma, Leo Luo, Yang Li, Chengming Li, Peng Chen, Jie Jiang
Computation and Language · Cross-listed from cs.AI
The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models
Chonghan Qin, Xiachong Feng, Ziyun Song et al. (+3 more)
Large
language models (LLMs) are routinely prompted to take on social roles ranging from individuals to institutions, yet it remains unclear whether their internal representations encode the granularity of such roles, from micro-level individual experience to macro-level organizational, institutional, or national
reasoning. We show that they do. We define a contrast-based Granularity Axis as the difference between mean macro- and micro-role hidden states. In Qwen3-8B, this axis aligns with the principal axis (PC1) of the role representation space at cosine 0.972 and accounts for 52.6% of its variance, indicating that granularity is the dominant
geometric axis organizing prompted social roles. We construct 75 social roles across five granularity levels and collect 91,200 role-conditioned responses over shared questions and
prompt variants, then extract role-level hidden states and project them onto the axis. Role projections increase monotonically across all five levels, remain stable across layers,
prompt variants, endpoint definitions, held-out splits, and score-filtered subsets, and transfer to Llama-3.1-8B-Instruct. The axis is also causally relevant: activation steering along it shifts response granularity in the predicted direction, with Llama moving from 2.00 to 3.17 on a five-point macro scale under positive steering on
prompts that admit local responses. The two models differ in controllability, suggesting that steering depends on each model's default operating regime. Overall, our findings suggest that social role granularity is not merely a stylistic surface feature, but a structured, ordered, and causally manipulable latent direction in role-conditioned
language model behavior.
Read the full abstract ▾
All authors Chonghan Qin, Xiachong Feng, Ziyun Song, Xiaocheng Feng, Jing Xiong, Lingpeng Kong
Computation and Language · Cross-listed from cs.AI
Hindsight Compacts but Does Not Repair: Rethinking On-Policy Self-Distillation in Reasoning Models
Jaehoon Kim and Dongha Lee
On-Policy Self-
Distillation (OPSD) has emerged as a promising post-training method: requiring only privileged context, it enables
token-level credit assignment from a self-teacher with hindsight, yielding higher accuracy with shorter responses. However, in thinking-enabled mathematical
reasoning, where solving a problem takes long
reasoning traces that explore, reflect, and backtrack, its length reductions persist but its accuracy gains largely do not. Does OPSD's hindsight signal, then, still repair failed trajectories, or does it only compact viable ones? We hypothesize compaction: in long traces, hindsight reveals which steps were unnecessary more readily than which steps would have repaired the solution. To test this, we apply OPSD separately to correct and incorrect rollouts. Training only on correct rollouts shortens responses by 18-29% while largely preserving accuracy, whereas training only on incorrect rollouts degrades it. The accuracy gap holds across three models, six
benchmarks, and three seeds, and under a same-
prompt control for difficulty. Neither branch raises the pass@k ceiling, and both suppress exploration and reflection markers, which viable traces can spare but failed ones need. Changing the divergence, enriching or reinjecting the privileged context, and training longer only move OPSD along the same accuracy-length tradeoff. Hindsight compacts
reasoning the model can already produce but does not repair
reasoning it cannot.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Rethinking Adapter Placement: A Dominant Adaptation Module Perspective
Suoxin Zhang, Run He, Di Fang et al. (+3 more)
Low-rank adaptation (LoRA) is a widely used parameter-efficient fine-tuning method that places trainable low-rank adapters into frozen pre-trained models. Recent studies show that using fewer LoRA adapters may still maintain or even improve performance, but existing methods still distribute adapters broadly, leaving where to place a limited number of adapters to maximize performance largely open. To investigate this, we introduce PAGE (Projected Adapter Gradient Energy), a gradient-based sensitivity probe that estimates the initial trainable gradient energy available to each candidate LoRA adapter. Surprisingly, we find that PAGE is highly concentrated on a single shallow FFN down-projection across two model families and four downstream tasks. We term this module the dominant adaptation module and show that its layer index is architecture-dependent but task-stable. Motivated by this finding, we propose DomLoRA, a placement method that places a single adapter at the dominant adaptation module. With only 0.7% of vanilla LoRA's trainable parameters, DomLoRA outperforms it on average across downstream tasks, including instruction following, mathematical
reasoning, coding, and multi-turn conversation. This method also matches or improves other LoRA variants and reduces training time by up to 2.74$\times$ compared with broad placement, supporting the dominant adaptation module perspective as a practical placement guideline.
Read the full abstract ▾
All authors Suoxin Zhang, Run He, Di Fang, Xiang Tan, Kaixuan Chen, Huiping Zhuang
Computation and Language · Cross-listed from cs.LG
Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes
Hanqing Liu, Jianjun Cao, Yuanze Li et al. (+1 more)
Deep neural networks exhibit periodic loss spikes during unregularized long-term training, a phenomenon known as the "Slingshot Mechanism." Existing work usually attributes this to intrinsic optimization dynamics, but its triggering mechanism remains unclear. This paper proves that this phenomenon is a result of floating-point arithmetic precision limits. As training enters a high-confidence stage, the difference between the correct-class logit and the other logits may exceed the absorption-error threshold. Then during backpropagation, the gradient of the correct class is rounded exactly to zero, while the gradients of the incorrect classes remain nonzero. This breaks the zero-sum constraint of gradients across classes and introduces a systematic drift in the parameter update of the classifier layer. We prove that this drift forms a positive feedback loop with the feature, causing the global classifier mean and the global feature mean to grow exponentially. We call this mechanism Numerical Feature Inflation (NFI). This mechanism explains the rapid norm growth before a Slingshot spike, the subsequent reappearance of gradients, and the resulting loss spike. We further show that NFI is not equivalent to an observed loss spike: in more practical tasks, partial absorption may not produce visible spikes, but it can still break the zero-sum constraint and drive rapid growth of parameter norms. Our results reinterpret Slingshot as a numerical dynamic of finite-precision training, and provide a testable explanation for abnormal parameter growth and logit divergence in late-stage training.
Read the full abstract ▾
All authors Hanqing Liu, Jianjun Cao, Yuanze Li, Zijian Zhou
Computation and Language
IRC-Bench: Recognizing Entities from Contextual Cues in First-Person Reminiscences
Yehudit Aperstein, Eden Moran and Alexander Apartsin
When people recount personal
memories, they often refer to people, places, and events indirectly, relying on con-textual cues rather than explicit names. Such implicit references are central to reminiscence narratives: first-person accounts of lived experience used in therapeutic, archival, and social settings. They pose a difficult computational problem because the intended entity must be inferred from dispersed narrative evidence rather than from a local mention. We introduce IRC-Bench, the Implicit Reminiscence Context
Benchmark, for evaluating implicit entity recognition in reminiscence transcripts. The
benchmark targets non-locality: entity-identifying cues are distributed across multiple, non-contiguous clauses, unlike named entity recognition, entity linking, or coreference resolution. IRC-Bench comprises 25,136 samples constructed from 12,337 Wikidata-linked entities across 1,994 transcripts spanning 11 thematic domains. Each sample pairs an Entity-Grounded Narrative, in which the target entity is ex-plicitly mentioned, with an Entity-Elided Narrative, in which direct mentions are removed. We evaluate 19 config-urations across LLM generation, dense
retrieval, RAG, and fine-tuning. QLoRA-adapted Llama 3.1 8B performs best in the open-world setting (38.94% exact match; 51.59% Jaccard), while fine-tuned DPR leads closed-world
retrieval (35.38% Hit@1; 71.49% Hit@10). We release IRC-Bench with data, code, and evaluation tools.
Read the full abstract ▾
Computation and Language
Chunyu Li, Mengyuan Zhang, Jingyi Kang et al. (+6 more)
In
agent memory systems, the reranking model serves as the critical bridge connecting user queries with long-term
memory. Most systems adopt the "retrieve-then-rerank" two-stage paradigm, but generic reranking models rely on semantic similarity matching and lack genuine
reasoning capabilities, leading to a problem where recalled results are semantically highly relevant yet do not contain the key information needed to answer the question. This deficiency manifests in
memory scenarios as three specific problems. First, relevance scores are miscalibrated, making threshold-based filtering difficult. Second, ranking degrades when facing temporal constraints, causal
reasoning, and other complex queries. Third, the model cannot leverage dialogue context for semantic disambiguation. This report introduces MemReranker, a reranking model family (0.6B/4B) built on Qwen3-Reranker through multi-stage LLM knowledge
distillation. Multi-teacher pairwise comparisons generate calibrated soft labels, BCE pointwise
distillation establishes well-distributed scores, and InfoNCE contrastive learning enhances hard-sample discrimination. Training data combines general corpora with
memory-specific multi-turn dialogue data covering temporal constraints, causal
reasoning, and coreference resolution. On the
memory retrieval benchmark, MemReranker-0.6B substantially outperforms BGE-Reranker and matches open-source 4B/8B models as well as GPT-4o-mini on key metrics. MemReranker-4B further achieves 0.737 MAP, with several metrics on par with Gemini-3-Flash, while maintaining inference latency at only 10–20% of large models. On finance and healthcare vertical-domain
benchmarks, the models preserve generalization capabilities on par with mainstream large-parameter rerankers.
Read the full abstract ▾
All authors Chunyu Li, Mengyuan Zhang, Jingyi Kang, Ding Chen, Jiajun Shen, Bo Tang, Xuanhe Zhou, Feiyu Xiong, Zhiyu Li
Computation and Language · Cross-listed from cs.AI
On Time, Within Budget: Constraint-Driven Online Resource Allocation for Agentic Workflows
Xinglin Wang, Zishen Liu, Shaoxiong Feng et al. (+9 more)
Agentic systems increasingly solve complex user requests by executing orchestrated workflows, where subtasks are assigned to specialized models or tools and coordinated according to their dependencies. While recent work improves
agent efficiency by optimizing the performance–cost–latency frontier, real deployments often impose concrete requirements: a workflow must be completed within a specified budget and before a specified deadline. This shifts the goal from average efficiency optimization to maximizing the probability that the entire workflow completes successfully under explicit budget and deadline constraints. We study constraint-driven online resource allocation for agentic workflows. Given a dependency-structured workflow and estimates of success rates and generation lengths for each subtask–model pair, the executor dynamically allocates models and parallel samples across simultaneously executable subtasks while managing the remaining budget and time. We formulate this setting as a finite-horizon
stochastic online allocation problem and propose Monte Carlo Portfolio Planning (MCPP), a lightweight closed-loop planner that directly estimates constrained completion probability through simulated workflow executions and replans after observed outcomes. Experiments on CodeFlow and ProofFlow demonstrate that MCPP consistently improves constrained completion probability over strong baselines across a wide range of budget–deadline constraints.
Read the full abstract ▾
All authors Xinglin Wang, Zishen Liu, Shaoxiong Feng, Peiwen Yuan, Yiwei Li, Jiayi Shi, Yueqi Zhang, Chuyi Tan, Ji Zhang, Boyuan Pan, Yao Hu, Kan Li
Computation and Language
Uncovering Entity Identity Confusion in Multimodal Knowledge Editing
Shu Wu, Xiaotian Ye, Xinyu Mou et al. (+3 more)
Multimodal knowledge editing (MKE) aims to correct the internal knowledge of large
vision-
language models after deployment, yet the behavioral patterns of post-edit models remain underexplored. In this paper, we identify a systemic failure mode in edited models, termed Entity Identity Confusion (EIC): edited models exhibit an absurd behavior where text-only queries about the original entity's identity unexpectedly return information about the new entity. To rigorously investigate EIC, we construct EC-Bench, a diagnostic
benchmark that directly probes how image-entity bindings shift before and after editing. Our analysis reveals that EIC stems from existing methods failing to distinguish between Image-Entity (I-E) binding and Entity-Entity (E-E) relational knowledge in the model, causing models to overfit E-E associations as a shortcut: the image is still perceived as the original entity, with the new entity's name serving only as a spurious identity label. We further explore potential mitigation strategies, showing that constraining edits to the model's I-E processing stage encourages edits to act more faithfully on I-E binding, thereby substantially reducing EIC. Based on these findings, we discuss principled desiderata for faithful MKE and provide methodological guidance for future research.
Read the full abstract ▾
All authors Shu Wu, Xiaotian Ye, Xinyu Mou, Dongsheng Liu, Xiaohan Wang, Mengqi Zhang
Computation and Language
Milestone-Guided Policy Learning for Long-Horizon Language Agents
Zixuan Wang, Yuchen Yan, Hongxing Li et al. (+7 more)
While long-horizon agentic tasks require
language agents to perform dozens of sequential decisions, training such
agents with reinforcement learning remains challenging. We identify two root causes: credit misattribution, where correct early actions are penalized due to terminal failures, and sample inefficiency, where scarce successful trajectories result in near-total loss of learning signal. We introduce a milestone-guided policy learning framework, BEACON, that leverages the compositional structure of long-horizon tasks to ensure precise credit assignment. BEACON partitions trajectories at milestone boundaries, applies temporal reward shaping within segments to credit partial progress, and estimates advantages at dual scales to prevent distant failures from corrupting the evaluation of local actions. On ALFWorld, WebShop, and ScienceWorld, BEACON consistently outperforms GRPO and GiGPO. Notably, on long-horizon ALFWorld tasks, BEACON achieves 92.9% success rate, nearly doubling GRPO's 53.5%, while improving effective sample utilization from 23.7% to 82.0%. These results establish milestone-anchored credit assignment as an effective paradigm for training long-horizon
language agents. Code is available at https://github.com/ZJU-REAL/BEACON.
Read the full abstract ▾
All authors Zixuan Wang, Yuchen Yan, Hongxing Li, Teng Pan, Dingming Li, Ruiqing Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
Computation and Language
Navigating by Old Maps: The Pitfalls of Static Mechanistic Localization in LLM Post-Training
Hang Chen, Jiaying Zhu, Hongyang Chen et al. (+3 more)
The "Locate-then-Update" paradigm has become a predominant approach in the post-training of large
language models (LLMs), identifying critical components via mechanistic interpretability for targeted parameter updates. However, this paradigm rests on a fundamental yet unverified assumption: can mechanisms derived from current static parameters reliably guide future dynamic parameter updates? To investigate this, we systematically track the structural
evolution of
Transformer circuits throughout the supervised fine-tuning (SFT) process, revealing the underlying dynamics of task mechanisms. We introduce three novel metrics-
Circuit Distance,
Circuit Stability, and
Circuit Conflict-to analyze
circuit evolution across three dimensions: neural migration, semantic stability, and cross-task interference. Our empirical results reveal that
circuits inherently exhibit "Free
Evolution" during parameter updates. Consequently, static mechanisms extracted from current states inevitably suffer from temporal latency, making them fundamentally inadequate for guiding future states. Moreover, by deconstructing the "illusion of effectiveness" in existing methods, this work underscores the necessity of "foresight" in mechanistic localization and proposes a predictive framework for future research.
Read the full abstract ▾
All authors Hang Chen, Jiaying Zhu, Hongyang Chen, Hongxu Liu, Xinyu Yang, Wenya Wang
Computation and Language · Cross-listed from cs.AI
Novelty-based Tree-of-Thought Search for LLM Reasoning and Planning
Leon Hamm and Zlatan Ajanovic
Although advances such as chain-of-thought, tree-of-thought or reinforcement learning have improved the performance of LLMs in
reasoning and planning tasks, they are still brittle and have not achieved human-level performance in many domains, and often suffer from high time and
token costs. Inspired by the success of width-based search in planning, we explore how the concept of novelty can be transferred to
language domains and how it can improve tree-of-thought
reasoning. A tree of thoughts relies on building possible "paths" of consecutive ideas or thoughts. These are generated by repeatedly prompting an LLM. In our paper, a measurable concept of novelty is proposed that describes the uniqueness of a new node (thought) in comparison to nodes previously seen in the search tree. Novelty is estimated by prompting an LLM and making use of embedded general knowledge from pre-training. This metric can then be used to prune branches and reduce the scope of the search. Although this method introduces more
prompts per state, the overall
token cost can be reduced by
pruning and reducing the overall tree size. This procedure is tested and compared using several
benchmarks in
language-based planning and general
reasoning.
Read the full abstract ▾
Computation and Language
More Aligned, Less Diverse? Analyzing the Grammar and Lexicon of Two Generations of LLMs
Adrián Gude, Roi Santos-Ríos, Francis Bond et al. (+3 more)
This study contributes to a growing line of research in comparing LLM-generated texts with human-authored text, in this case, English news text. We focus in particular on the evaluation of syntactic properties through formal grammar frameworks. Our analysis compares two generations of LLMs in the context of two human-authored English news
datasets from two different years. Employing the Head-Driven Phrase Structure Grammar (HPSG) formalism, we investigate the distributions of syntactic structures and lexical types of AI-generated texts and contrast them with the corresponding distributions in the human-authored New York Times (NYT) articles. We use diversity metrics from ecology and information theory to quantify variation in grammatical constructions and lexical types. We show that English news text has changed little in the given time frame, while newer LLMs display reduced syntactic and, especially, lexical diversity compared to older, non-instruction-tuned models. These findings point to future work in studying effects of instruction tuning, which, while enhancing coherence and adherence to
prompts, may narrow the expressive range of model output.
Read the full abstract ▾
All authors Adrián Gude, Roi Santos-Ríos, Francis Bond, Dan Flickinger, Carlos Gómez-Rodríguez, Olga Zamaraeva
Computation and Language
PersonaKit (PK): A Plug-and-Play Platform for User Testing Diverse Roles in Full-Duplex Dialogue
Hyunbae Jeon and Jinho D. Choi
As spoken dialogue systems expand beyond traditional assistant roles to encompass diverse personas – such as authoritative instructors, uncooperative merchants, or distracted workers – they require distinct, human-like turn-taking behaviors to maintain psychological immersion. However, current full-duplex systems often default to a rigid, overly accommodating “always-yield” policy during overlapping
speech, which severely undermines character consistency for non-submissive roles. Evaluating alternative, persona-specific turn-taking strategies through empirical user studies is challenging because building real-time full-duplex test environments requires substantial engineering overhead. To address this, we present PersonaKit (PK), an open-source, low-latency web platform for the rapid prototyping and evaluation of conversational
agents. Using intuitive JSON configurations, researchers can define personas, specify probabilistic interruption-handling behaviors (e.g., yield, hold, bridge, or override), and automatically deploy comparative A/B surveys. Through an in-the-wild evaluation with 8 distinct personas, we demonstrate that PersonaKit provides an extensible, end-to-end framework for studying complex sociolinguistic behaviors in next-generation spoken
agents.
Read the full abstract ▾
Computation and Language
From Articles to Premises: Building PrimeFacts, an Extraction Methodology and Resource for Fact-Checking Evidence
Premtim Sahitaj, Jawan Kolanowski, Ariana Sahitaj et al. (+7 more)
Fact-checking articles encode rich supporting evidence and
reasoning, yet this evidence remains largely inaccessible to automated verification systems due to unstructured presentation. We introduce PrimeFacts, a methodology and resource for extracting fine-grained evidence from full fact-checking articles. We compile 13,106 PolitiFact articles with claims, verdicts, and all referenced sources, and we identify 49,718 in-article hyperlinks as natural anchors to pinpoint key evidence. Our framework leverages large
language models (LLMs) to rewrite these anchor sentences into stand-alone, context-independent premises and investigates the extraction of additional implicit evidence. In evaluations on cross-article evidence
retrieval and claim verification, the extracted premises substantially improve performance. Decontextualized evidence yields higher retrievability, achieving up to a 30 percent relative gain in Mean Reciprocal Rank over verbatim sentences, and using the evidence for verdict prediction raises Macro-F1 by 10-20 points over the baseline. These gains are consistent across different verdict granularities (2-class vs. 5-class) and model architectures. A qualitative analysis indicates that the decontextualized premises remain faithful to the original sources. Our work highlights the promise of reusing fact-checkers' evidence for automation and provides a large-scale resource of structured evidence from real-world fact-checks.
Read the full abstract ▾
All authors Premtim Sahitaj, Jawan Kolanowski, Ariana Sahitaj, Veronika Solopova, Max Upravitelev, Daniel Röder, Iffat Maab, Junichi Yamagishi, Sebastian Möller, Vera Schmitt
Computation and Language · Cross-listed from cs.CR
Safety Anchor: Defending Harmful Fine-tuning via Geometric Bottlenecks
Guoxin Lu, Letian Sha, Qing Wang et al. (+4 more)
The safety
alignment of Large
Language Models (LLMs) remains vulnerable to Harmful Fine-tuning (HFT). While existing defenses impose constraints on parameters, gradients, or internal representations, we observe that they can be effectively circumvented under persistent HFT. Our analysis traces this failure to the inherent redundancy of the high-dimensional parameter space: attackers exploit optimization trajectories that are orthogonal to defense constraints to restore harmful capabilities while deceptively adhering to safety restrictions. To address this, we propose Safety Bottleneck Regularization (SBR). SBR shifts the defensive focus from the redundant parameter space to the unembedding layer, which serves as a
geometric bottleneck. By anchoring the final hidden states of harmful queries to those of the safety-aligned model, SBR enables the model to maintain safe responses even under persistent HFT. Extensive experiments confirm SBR's effectiveness, demonstrating that utilizing just a single safety anchor is sufficient to reduce the Harmful Score to $<$10 while preserving competitive performance on benign downstream tasks.
Read the full abstract ▾
All authors Guoxin Lu, Letian Sha, Qing Wang, Peijie Sun, Hao Zhou, Hua Dai, Fu Xiao
Computation and Language · Cross-listed from cs.AI
TheraAgent: Self-Improving Therapeutic Agent for Precise and Comprehensive Treatment Planning
Junkai Li, Yunghwei Lai, Tianyi Zhu et al. (+3 more)
Formulating a treatment plan is inherently a complex
reasoning and refinement task rather than a simple generation problem. However, existing large
language models (LLMs) mainly rely on one-shot output without explicit verification, which may result in rough, incomplete, and potentially unsafe treatment plans. To address these limitations, we propose TheraAgent, an agentic framework that replaces one-shot generation with an iterative generate-judge-refine pipeline. By mirroring the actual
reasoning process of human experts who iteratively revise treatment plans, our framework progressively transforms coarse and incomplete drafts into precise, comprehensive, and safer therapeutic regimens. To facilitate the critical judge component, we introduce TheraJudge, a treatment-specific evaluation module integrated into the inference loop to enforce clinical standards. Experiments show TheraAgent achieves state-of-the-art results on HealthBench, leading in Accuracy and Completeness. In expert evaluations, it attains an 86% win rate against physicians, with superior Targeting and Harm Control. Moreover, the highly agreement between TheraJudge and HealthBench evaluations confirms the reliability of our framework.
Read the full abstract ▾
All authors Junkai Li, Yunghwei Lai, Tianyi Zhu, Zheng Long Lee, Weizhi Ma, Yang Liu
Computation and Language
Tatarstan Toponyms: A Bilingual Dataset and Hybrid RAG System for Geospatial Question Answering
Mullosharaf K. Arabov, Svetlana S. Khaybullina and Dinara M. Naumetova
This paper addresses end-to-end geospatial question answering over
multilingual toponymic data. We introduce a bilingual (Russian-Tatar)
dataset of 9,688 toponyms with linguistic, etymological, and coordinate information (93.1 percent georeferenced). Based on this, we construct about 39,000 question-context-answer triples with guaranteed answer localization. Our architecture combines a hybrid retriever (dense semantic indexing with
multilingual-e5-large plus geospatial filtering/ranking using KD-trees and haversine distance) and an extractive reader fine-tuned on
transformer models. On 500 test queries, hybrid search achieves Recall@1 = 0.988, Recall@5 = 1.000, MRR = 0.994, significantly outperforming BM25 and spatial-only methods. Among readers (RuBERT, XLM-RoBERTa-large, T5-RUS), XLM-RoBERTa-large gives best results: EM = 0.992, F1 = 0.994. RuBERT models fail on coordinate questions due to tokenization artifacts, but simple post-processing recovers 100 percent accuracy. Resources (
dataset, QA corpus, models, web demo) are openly released on Hugging Face. Results are directly applicable to geospatial QA services, geocoding, and digital humanities projects.
Read the full abstract ▾
Computation and Language
TableVista: Benchmarking Multimodal Table Reasoning under Visual and Structural Complexity
Zheyuan Yang, Liqiang Shang, Junjie Chen et al. (+6 more)
We introduce TableVista, a comprehensive
benchmark for evaluating foundation models in
multimodal table
reasoning under visual and structural complexity. TableVista consists of 3,000 high-quality table
reasoning problems, where each instance is expanded into 10 distinct visual variants through our multi-style rendering and transformation pipeline. This process encompasses diverse scenario styles, robustness perturbations, and
vision-only configurations, culminating in 30,000
multimodal samples for a multi-dimensional evaluation. We conduct an extensive evaluation of 29 state-of-the-art open-source and proprietary foundation models on TableVista. Through comprehensive quantitative and qualitative analysis, we find that while evaluated models remain largely stable across diverse rendering styles, they exhibit pronounced performance degradation on complex structural layouts and
vision-only settings, revealing that current models struggle to maintain
reasoning consistency when structural complexity combines with visually integrated presentations. These findings highlight critical gaps in current
multimodal capabilities, providing insights for advancing more robust and reliable table understanding models.
Read the full abstract ▾
All authors Zheyuan Yang, Liqiang Shang, Junjie Chen, Xun Yang, Chenglong Xu, Bo Yuan, Chenyuan Jiao, Yaoru Sun, Yilun Zhao
Computation and Language
Hallucination as an Anomaly: Dynamic Intervention via Probabilistic Circuits
Erik Nielsen, Elia Cunegatti, Marcus Vukojevic et al. (+1 more)
One of the most critical challenges in Large
Language Models is their tendency to hallucinate, i.e., produce factually incorrect responses. Existing approaches show promising results in terms of
hallucination correction, but still suffer from a main limitation: they apply corrections indiscriminately to every
token, corrupting also the originally correct generations. To overcome this drawback, we propose PCNET, a Probabilistic
Circuit trained as a tractable density
estimator over the LLM residual stream. The method detects
hallucinations as
geometric anomalies on the factual
manifold, which is done via exact Negative Log-Likelihood computation, hence without the need for sampling, external verifiers, or weight modifications, as in existing techniques. To demonstrate its effectiveness, we exploit PCNET as a dynamic gate that distinguishes hallucinated from factual hidden states at each decoding step. This triggers our second main contribution, PC-LDCD (Probabilistic
Circuit Latent Density Contrastive Decoding), only when the latent geometry deviates from factual regions, while leaving correct generations untouched. Across four LLMs, ranging from 1B to 8B models, and four
benchmarks covering conversational
reasoning, knowledge-intensive QA, reading comprehension, and truthfulness, PCNET achieves near-perfect
hallucination detection across CoQA, SQuAD v2.0, and TriviaQA, with AUROC reaching up to 99%. Moreover, PC-LDCD obtains the highest True+Info, MC2, and MC3 scores on TruthfulQA in three out of four models, in comparison with state-of-the-art baselines, while reducing the mean corruption rate to 53.7% and achieving a preservation rate of 79.3%. Our proposed method is publicly available on GitHub.
Read the full abstract ▾
All authors Erik Nielsen, Elia Cunegatti, Marcus Vukojevic, Giovanni Iacca
Computation and Language
Lightweight Stylistic Consistency Profiling: Robust Detection of LLM-Generated Textual Content for Multimedia Moderation
Siyuan Li, Aodu Wulianghai, Xi Lin et al. (+6 more)
The increasing prevalence of Large
Language Models (LLMs) in content creation has made distinguishing human-written textual content from LLM-generated counterparts a critical task for multimedia moderation. Existing detectors often rely on statistical cues or model-specific heuristics, making them vulnerable to paraphrasing and
adversarial manipulations, and consequently limiting their robustness and interpretability. In this work, we proposeLiSCP , a novel lightweight stylistic consistency profiling method for robust detection of LLM-generated textual content, focusing on feature stability under
adversarial manipulation. Our approach constructs a consistency profile that combines discrete stylistic features with continuous semantic signals, leveraging stylistic stability across
multimodal-guided paraphrased text variants. Experiments spanning real-world multimedia news and movie
datasets and conventional text domains demonstrate that LiSCP achieves superior performance on in-domain detection and outperforms existing approaches by up to 11.79% in cross-domain settings. Additionally,it demonstrates notable robustness under
adversarial scenarios, including
adversarial attacks and hybrid human-AI settings.
Read the full abstract ▾
All authors Siyuan Li, Aodu Wulianghai, Xi Lin, Xibin Yuan, Qinghua Mao, Guangyan Li, Xiang Chen, Jun Wu, Jianhua Li
Computation and Language · Cross-listed from cs.CV
MobileEgo Anywhere: Open Infrastructure for long horizon egocentric data on commodity hardware
Senthil Palanisamy, Abhishek Anand, Satpal Singh Rathore et al. (+3 more)
Vision-
language-action (VLA) models have driven demand for large-scale
egocentric datasets, yet the hardware and infrastructure to collect long-horizon data remain inaccessible.
Datasets today typically have episodes only a few minutes long, which fails to capture the long-horizon temporal dependencies that complex robotic task execution requires. We present MobileEgo Anywhere, a framework for collecting hour-plus
egocentric trajectories on commodity mobile hardware that uses modern smartphone sensors for long-term pose tracking without the hardware barriers of traditional robotics data collection. We release three components: (1) STERA, an open-source
video-processing pipeline that converts raw mobile captures into standardized, training-ready formats for VLA and foundation-model research; (2) a free mobile app that lets any user record
egocentric activity; and (3) a 200-hour
dataset of diverse, long-form
egocentric data with persistent state tracking across 584 sessions. We further show this data is a usable training signal:mid-training a VLA on it lowers held-out action-prediction error.
Read the full abstract ▾
All authors Senthil Palanisamy, Abhishek Anand, Satpal Singh Rathore, Pratyush Patnaik, Shubhanshu Khatana, Ekaksh Janweja
Computation and Language · Cross-listed from cs.LG
Near-Policy: Accelerating On-Policy Distillation via Asynchronous Generation and Selective Packing
Miao Rang, Zhenni Bi, Hang Zhou et al. (+6 more)
Standard knowledge
distillation for autoregressive models often suffers from distribution mismatch. While on-policy methods mitigate this by leveraging student-generated outputs, they rely on computationally expensive Reinforcement Learning (RL) frameworks. To improve efficiency, we propose Near-Policy
Distillation (NPD), an asynchronous approach that decouples student generation from training. This reformulation enables Supervised Fine-Tuning (SFT) with sequence packing. However, asynchronous updates inevitably introduce policy lag and sample noise, which can cause the behavior to drift from near-policy toward off-policy. To counteract this without sacrificing efficiency, NPD integrates sparse student updates and the $\Delta$-IFD filtering mechanism, a heuristic sample selection mechanism that empirically stabilizes the optimization trajectory. By filtering extreme out-of-distribution samples, $\Delta$-IFD prevents noise from dominating the gradients, ensuring updates remain within a safe proximal learning zone. Empirically, the NPD framework achieves a 8.1x speedup over on-policy baselines and outperforms SFT by 8.09%. Crucially, by effectively narrowing the exploration space for subsequent RL, our method enables openPangu-Embedded-1B to reach a state-of-the-art score of 68.73%, outperforming the substantially larger Qwen3-1.7B. Codes will be released soon.
Read the full abstract ▾
All authors Miao Rang, Zhenni Bi, Hang Zhou, Kai Han, Xuechun Wang, An Xiao, Xinghao Chen, Yunhe Wang, Hanting Chen
Computation and Language
Minimizing Modality Gap from the Input Side: Your Speech LLM Can Be a Prosody-Aware Text LLM
Wenqian Cui, Xiao-Hui Li, Daxin Tan et al. (+2 more)
Speech large
language models (SLMs) are typically built from text large
language model (TLM) checkpoints, yet they still suffer from a substantial modality gap. Prior work has mainly attempted to reduce this gap from the output side by making
speech generation more text-like, but the gap remains. We argue that the key remaining bottleneck lies on the input side. We propose TextPro-SLM, an SLM that makes spoken input more closely resemble that of a prosody-aware text LLM. TextPro-SLM combines WhisperPro, a unified
speech encoder that produces synchronized text
tokens and prosody
embeddings, with an LLM backbone trained to preserve the semantic capabilities of the original TLM while learning paralinguistic understanding. Experiments show that TextPro-SLM achieves the lowest modality gap among leading SLMs at both 3B and 7B scales, while also delivering strong overall performance on paralinguistic understanding tasks. These gains are achieved with only roughly 1,000 hours of LLM training audio, suggesting that reducing the modality gap from the input side is both effective and data-efficient. Code is available at https://github.com/dreamtheater123/TextPro-SLM
Read the full abstract ▾
All authors Wenqian Cui, Xiao-Hui Li, Daxin Tan, Qiyong Zheng, Irwin King