Computation and Language · Cross-listed from cs.LG
Self-Evolution for Multi-Turn Tool-Calling Agents via Divergence-Point Preference Learning
Jiaqiang Tang
Multi-turn tool-using
agents must coordinate long-horizon tool sequences while tracking dialogue state and policy constraints. Existing approaches often separate inference-time orchestration from parameter-level learning, leaving tool selection weakly structured and preference updates vulnerable to train–deployment
prompt mismatch. For within-
benchmark self-improvement, ToolGraph combines schema-derived
topology, transition weights estimated from successful rollouts, and history-aware controls for write prerequisites and repeated-search loops. We then construct 161 preference pairs by locating divergence points via state-based matching and prefix-based
alignment, filtered through action-correctness annotations, and train DPO under the same ToolGraph context used at inference. Across 375 tau2-bench tasks, ToolGraph raises the weighted average reward from 0.304 to 0.338 (+11.2% relative), while ToolGraph+DPO reaches 0.355 (+16.8% over the baseline), with the DPO gain concentrated in airline and retail. Fine-grained diagnostics further show that roughly half of telecom trajectories exhaust the step budget before action execution and that chosen reward positivity is the most useful checkpoint signal across our 16 evaluated DPO configurations.
Read the full abstract ▾
Computation and Language
A Dual-Track Framework for Template-Constrained LaTeX Conversion
Chung Cheuk Hei and Liu Li
With the increasing demands for advanced document conversion, mapping structured Markdown drafts into template-compliant formats like LaTeX remains a challenge. Existing approaches largely depend on either deterministic rule-based converters or pure end-to-end Large
Language Model (LLM) generation. The former fails to correctly handle asset insertions and template-specific constraints, while the latter tends to induce semantic drift, leading to
hallucinations that are difficult to debug. To address these limitations, we introduce a robust Dual-Track Framework that systematically decouples template formatting from document processing: an offline track extracts template constraints into a reusable manifest, while an online track implements a hybrid execution pipeline. This pipeline confines LLM usage exclusively to
reasoning-intensive components (e.g., semantic metadata, bibliographic references, and complex visual/tabular layouts) while delegating rule-based engines for deterministic processing. Empirical evaluation across 7 LaTeX templates and 56 published research papers demonstrates that our method preserves better structural fidelity, satisfies diverse layout constraints, and achieves a higher compilation success rate compared to the previous baselines.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Cognitive Digital Twins: Ethical Risks and Governance for AI Systems That Model the Mind
Vamshi Krishna Bonagiri, Juan Nicolas Sepulveda-Arias, Abdoul Jalil Djiberou Mahamadou et al. (+1 more)
As AI systems become increasingly persistent and personalized, they make possible a class of technologies that we call cognitive digital twins (CDTs): dynamic computational representations of a specific person's cognition, updated from behavioral, contextual, or physiological data in order to model, predict, or simulate that person's cognition, or to act as that person's communicative or decision-making proxy. CDTs combine cognitive inference with longitudinal representation, simulation, and proxy action in ways that existing governance strategies for personal assistants, autonomous
agents, recommender systems, and automated decision systems only partially address. This paper makes four contributions. First, we define CDTs and distinguish them from adjacent systems. Second, we introduce a 5A governance framework organized around authority, autonomy, access and control, accountability, and availability. Third, we identify CDT-specific risks, from misrepresentation and epistemic authority shifts to shadow twins, simulated participation, proxy action, and proxy-power asymmetries. Fourth, we analyze governance gaps and propose requirements for high-risk CDTs that strengthen consent, purpose limitation, validity, traceability, contestation, independent review, and model retirement. Existing frameworks primarily regulate data processing, automated decisions, or autonomous actions; CDTs also require governance at the level of cognitive representation itself, before any final decision or external action occurs. We argue that CDTs require governance not only because they can act for people, but because they can become infrastructures through which cognition is represented, simulated, classified, and operationalized.
Read the full abstract ▾
All authors Vamshi Krishna Bonagiri, Juan Nicolas Sepulveda-Arias, Abdoul Jalil Djiberou Mahamadou, Monojit Choudhury
Computation and Language
PIVOTSBench: Evaluating Fine-Grained Interpersonal Relationship Reasoning in Multimodal Large Language Models
Shuxiang Zhang, Yiting Yin, Wenxuan Song et al. (+2 more)
Humans possess an innate ability to understand fine-grained interpersonal relationships, which is central to everyday social interactions. Although such
reasoning is inherently
multimodal, it remains largely unexplored by existing
multimodal large
language models (MLLMs). To address this gap, we introduce PIVOTS, the first
benchmark built from Social-IQ 2.0 and YouTube data to evaluate MLLMs' ability to predict bidirectional interpersonal relationship dimensions grounded in established psychology research. In addition, PIVOTS includes auxiliary tasks that assess models' ability to identify and leverage the critical visual cues underlying such predictions. We evaluate both proprietary and open-source MLLMs and conduct detailed ablation studies to analyze the effects of visual modalities and explicit social role information in conversational utterances. We further examine how joint and pairwise prediction settings benefit MLLMs in scoring bidirectional PIVOTS dimensions. Project page: https://ciossayin.github.io/pivots-bench/
Read the full abstract ▾
All authors Shuxiang Zhang, Yiting Yin, Wenxuan Song, Yuhang Wu, Miao Liu
Computation and Language · Cross-listed from cs.IR
Who Owns the AI Recommendation? A Multi-Industry Empirical Map of Brand Category Ownership Across Large Language Models
Dmitrij Żatuchin
This exploratory study measures brand inclusion across five industries, 50 brands and 250 queries, each put five times to GPT-5.2, Gemini 3 Flash and Perplexity sonar-pro in February and September 2026 (3,614 and 3,750 scored answers). Category Inclusion Rate, Recommendation Share, Competitive
Vacuum Index and Co-Mention Asymmetry have stated denominators. February inclusion rates sit close together across an industry's sampled brands (mean Gini 0.30), while at least one brand is named in 80% or more of answers to 204 of 250 queries. Vacuums occur in 7.6% of queries; provisional open-vocabulary model readings suggest most reflect the sampled brand list. The partially pre-specified September replication shows strong cross-date Recommendation Share correlation (Spearman 0.994), unchanged
vacuum prevalence and agreement of 60.8% against February's 57.2%. The descriptive size association persists. Fixed margins do not account for all co-mention structure: 31 ordered pairs depart from the September null. Agreement exceeds the query-independent null, and both parametric-pair advantage intervals are positive. All ten clustering seeds yield zero emergent clusters. Matcher validation remains provisional pending author checks. The statistics describe system output and identify no causal mechanism.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Unlimited OCR Works
Youyang Yin, Huanhuan Liu, YY et al. (+14 more)
Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large
language model (LLM) as the decoder allows the model to leverage the prior distribution of
language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up
memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working
memory. Taking DeepSeek OCR as the baseline, we replace all
attention layers in the decoder with our proposed Reference Sliding Window
Attention (R-SWA), which reduces
attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR's encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing
attention mechanism - beyond OCR, it is equally applicable to tasks such as ASR, translation, etc. Codes and model weights are publicly available at http://github.com/baidu/Unlimited-OCR.
Read the full abstract ▾
All authors Youyang Yin, Huanhuan Liu, YY, Qunyi Xie, Chaorun Liu, Shiqi Yang, Shaohua Wang, Zhanlong Liu, Hao Zou, Jinyue Chen, Shu Wei, Jingjing Wu, Mingxin Huang, Zhen Wu, Guibin Wang, Tengyu Du, Lei Jia
Computation and Language
PhoneBuddy: Training Open Models for Agentic Phone Use
Zhengyang Tang, Xin Lai, Pengyuan Lyu et al. (+23 more)
Phones are becoming an important execution surface for general-purpose
agents, but training open models for reliable phone use remains difficult because the environment that matters at deployment, real devices running real apps, is slow, stateful, side-effectful, and hard to reset or verify, while scalable mock environments only approximate real behavior. We present PhoneBuddy, a training recipe and open-model line for agentic phone use that combines a real-app environment with a mock-app environment, PhoneWorld, which reconstructs runnable mock apps from real GUI usage structure. PhoneBuddy first builds a shared supervised fine-tuning stage from trajectories collected in both environments, then compares real-app RL against mixed RL across both environments. Across a 150-task human evaluation on real phones spanning apps, mini-apps, and cross-app workflows, task success rate improves from 36.67% after supervised fine-tuning to 40.67% after real-app RL and 45.33% after mixed RL. On AndroidWorld, the same progression rises from 60.3% to 77.2% to 83.2%. These results show that mock-app training is not a replacement for real-app RL, but a complementary source of scalable, resettable, and automatically checked interaction. The gains are strongest on app and mini-app tasks, while long-horizontal cross-app workflows remain an important open challenge.
Read the full abstract ▾
All authors Zhengyang Tang, Xin Lai, Pengyuan Lyu, Xinyuan Wang, Tianyi Bai, Chenxin Li, Yiduo Guo, Huawen Shen, Yuxuan Liu, Junyi Li, Zhengyao Fang, Yang Ding, Yi Zhang, Weinong Wang, Xingran Zhou, Liang Wu, Fei Tang, Sunqi Fan, Shangpin Peng, Zheng Ruan, Anran Zhang, Benyou Wang, Ji-Rong Wen, Rui Yan, Chengquan Zhang, Han Hu
Computation and Language · Cross-listed from cs.CY
The Model as One Rater Among Several: Measuring Political Positions in Data-Sparse Regions with a Language-Model Panel
Tarek Gara
Most instruments for measuring political positions, such as manifesto coding, expert surveys and text-scaling models, were built and validated on Western party systems, and they break down worst where many consequential actors never publish a manifesto, never contest a competitive election, or commit one position to a charter and act on another in office. This paper proposes a method for such settings. I treat a large
language model as a single fallible rater in a panel, the way an expert survey treats one expert, and describe the panel, its synthesis, an applicability rule that keeps a blank distinct from a zero, and a lens that separates what an actor declares from what it does. Three results follow. First, a built-in experiment isolates the rubric: with a definition-free round held fixed, written axis definitions move scores by a mean of 1.8 points on a 21-point scale and tighten agreement between raters (mean absolute gap 2.81 to 2.50; r 0.81 to 0.89). Second, across nine models from eight laboratories in two countries, Krippendorff's alpha is 0.86 on an interval and 0.87 on an ordinal metric, and it held as the panel grew from five raters to nine. This is reliability and says nothing yet about validity, since models trained on overlapping data can share a bias no statistic computed from the models can detect. Third, the panel's sharpest split, a bimodal divergence on an actor's stance toward the state's foundational order, comes from a referent problem: the models agree on the facts and differ on which order counts as the regime. A blind triple-coding traces about two-thirds of such disagreement to interpretation. The worked example is the Middle East and North Africa, and the rubric,
prompt and data are released in full.
Read the full abstract ▾
Computation and Language
Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models
Yiran Zhu and Ziqi Yang
Large
Language Models (LLMs) raise growing concerns about
privacy leakage and copyright compliance. Membership inference is a key tool for assessing such risks, but existing studies mainly focus on whether specific samples or sample-based data units are used for training. We argue that LLMs exhibit a human-
memory-like behavior: an LLM may not memorize a specific sample verbatim, yet it can accumulate and reveal knowledge about a real-world entity from scattered mentions. This analogy motivates us to examine whether an LLM can be interrogated like a human interviewee to reveal its exposure to entity-related information. Motivated by this question, we propose entity-level membership inference, which determines whether information related to a target entity is used in LLM training. We study this task in the practical label-only black-box setting, where only generated texts are observable. We formalize the task under clue, input, and model constraints, establish the necessary and sufficient conditions for its feasibility, and instantiate five interrogation strategies based on this formalization. The strategies use limited entity clues to construct
prompts, elicit entity-related responses, and infer membership from semantic features among the generated texts. We construct entity-level
datasets and adapt state-of-the-art sample-level label-only methods to the entity-level setting as baselines. Experiments on person entities show that our methods achieve AUC up to 0.97 and bring gains of 6.0%–17.5% in Balanced Accuracy over the best adapted baseline.
Read the full abstract ▾
Computation and Language
Machine Translation and Post-Editing: Comparative Evaluation of Different MT Systems and Post-Editor Groups in Specialised Translation
Joachim Minder, Alexandra Mestivier and Natalie Kübler
This article aims to evaluate the quality of machine translation (MT) and post-editing (PE) in the context of specialised translation from English into French. Three MT systems (DeepL, eTranslation and Systran) were compared, and two groups of post-editors -linguists/translators and NLP experts -were asked to perform post-editing. Translation assessment is based on error annotation using an error typology adapted to MT and PE evaluation. The results reveal significant differences between the three MT systems and the two groups of post-editors, particularly in terms of terminological accuracy and fluency. This study highlights the importance of domain knowledge in specialised translation, as well as the limitations and variable performance of MT systems in
language for specific purposes (LSP).
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning
Yunan Wang, Minghui Song, Zihan Zhang et al. (+6 more)
Group-based Reinforcement Learning (RL) has significantly enhanced Large
Language Models (LLMs) in agentic scenarios. To achieve finer-grained policy updates, recent agentic RL frameworks have shifted from trajectory-level to step-level training. However, long-horizon agentic RL suffers from severe reward sparsity and delay, as feedback is often deferred for dozens of interaction steps. While existing step-level frameworks refine training granularity, their credit assignment remains coarse-grained and still treats
agent exploration as isolated, linear trajectories. This oversimplified perspective ignores the inherent graph structure of state transitions, leading to high-variance state-value estimation and myopic, localized credit assignment. To overcome these critical bottlenecks, we propose Group-Graph Policy Optimization (G2PO), a novel group-based RL
algorithm tailored for multi-turn agentic tasks. G2PO explicitly transforms linear interaction trajectories into a global state-transition graph. By aggregating identical observations across different trajectories, we introduce group-aggregation state-value estimation that reduces sampling variance and trajectory-dependent bias. Furthermore, we redefine
agent actions as transitions between state nodes and propose an edge-centric advantage estimation strategy. By globally standardizing Temporal Difference (TD) errors across the entire graph, G2PO explicitly identifies and prioritizes critical transitions that drive absolute task progress. Extensive experiments on representative long-horizon
benchmarks-WebShop, ALFWorld, and AppWorld-demonstrate that G2PO substantially outperforms state-of-the-art
prompt-based and RL baselines, achieving remarkable success rate improvements of up to 22.2% over GRPO.
Read the full abstract ▾
All authors Yunan Wang, Minghui Song, Zihan Zhang, Shaohan Huang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang
Computation and Language
Predicate Importance Estimation and Decoupled Rationale-Score Distillation for Entity Alignment
Keunha Kim, Yoonjin Jang, Hyeon-gu Lee et al. (+2 more)
Knowledge graphs (KGs) are increasingly used as structured context for Large
Language Models (LLMs), but industrial KG-RAG systems often need to integrate public and domain-specific KGs constructed from heterogeneous databases. This integration relies on Entity
Alignment (EA), where lexical matching alone is insufficient under predicate-name variation and incomplete local neighborhoods. We address EA for KG integration by constructing a pairwise EA
dataset and proposing two complementary modules: Predicate Importance Estimation (PIE) and Decoupled Rationale-Score
Distillation (DRSD). PIE is a compact
embedding-based approach that removes the subject information from each 1-hop triple, encodes the resulting subjectless triples, and aggregates them with learnable predicate-importance weights to build predicate-aware entity
embeddings. DRSD trains a distilled small
language model (SLM) with pseudo-answers produced by a teacher LLM through distinct
prompts. By converting binary EA labels into text-based supervision and decoupling confidence-score estimation from label-consistent rationales, DRSD enables the SLM to learn task-specific
reasoning while retaining a less label-biased confidence signal. Experiments show that PIE and DRSD improve EA classification. Moreover, because DRSD decouples confidence-score estimation from the decision, a discrepancy between the two flags an uncertain prediction for human review, thereby enabling a practical discrepancy between automatic acceptance and human-in-the-loop verification.
Read the full abstract ▾
All authors Keunha Kim, Yoonjin Jang, Hyeon-gu Lee, Sihyung Kim, Youngjoong Ko
Computation and Language
StatABench: Dataset and Framework for Evaluating Statistical Analysis Capabilities of LLMs
Youxin Zhu, Yixuan Ding, Peng Lai et al. (+3 more)
Statistical analysis is a broad, complex field requiring both domain knowledge and tool proficiency. While prior work has evaluated large
language models (LLMs) in this domain, existing
benchmarks remain limited in scope and format. To bridge this gap, we introduce StatABench (Statistical AnalysisBenchmark), a
benchmark designed to systematically assess LLMs' statistical analysis capabilities. StatABench comprises two complementary components: Stat-Closed, containing 404 questions across 18 statistical topics in multiple formats (multiple-choice, fill-in-the-blank, decision-making, and practical application), and Stat-Open, featuring 30 complex open-ended modeling tasks adapted from professional competitions. We evaluate diverse LLMs using the LangChain MCP framework and multiple data science
agents, and assess Stat-Open solutions via a validated LLM-as-Judge protocol. Experiments show that even GPT-5.1 achieves only 68.6% on Stat-Closed, while the best open-source model reaches 60.6%. On Stat-Open, the top
agent framework scores 61.86 on average. These results reveal the gap between current LLMs and reliable statistical analysis, highlighting persistent challenges in tool-grounded
reasoning, methodological decision-making, and end-to-end statistical modeling.
Read the full abstract ▾
All authors Youxin Zhu, Yixuan Ding, Peng Lai, Longyue Wang, Bingyi Jing, Guanhua Chen
Computation and Language · Cross-listed from cs.LG
Walk fast but be careful: Understanding Parallel Sampling in Masked Diffusion
Vansh Bansal, Cholyeon Cho, Syamantak Kumar et al. (+2 more)
In this paper, we use random walks on graphs as a verifiable sandbox for studying parallel sampling strategies in masked
diffusion models (MDMs). We train an MDM on random walk samples from a fixed graph. The graph and transition kernel are never shown to the model and serve as latent structure that is both controllable and enables evaluation. The framework provides a validity check for generated walks and a measure of distributional fidelity through the estimated transition kernel. Using simple graphs, we theoretically prove that parallel unmasking via widely used scores such as lowest
entropy is not uniformly better than random parallel sampling; even with exact conditional probabilities, performance critically depends on the conditional dependence structure induced by the graph, a phenomenon difficult to isolate in
benchmarks like Sudoku. We also develop training-free bisection samplers for MDMs, which take logarithmically many steps in the sequence length and are provably exact for random walks if the learned marginals are exact. Experiments on graph-walk tasks confirm that different parallel samplers perform better on different graph structures. Experiments on pretrained MDMs show that bisection-style samplers provide strong speed-quality tradeoffs on OpenWebText generation and
reasoning benchmarks including GSM8K, MBPP, and HumanEval. Together, these results use graph walks to uncover conditional dependence as a key principle of parallel MDM sampling and translate this insight into efficient samplers that transfer to
language generation and
reasoning.
Read the full abstract ▾
All authors Vansh Bansal, Cholyeon Cho, Syamantak Kumar, Sujay Sanghavi, Purnamrita Sarkar
Computation and Language · Cross-listed from cs.AI
Plans Fade Fast: Why Context Management Is Load-Bearing for LLM Agents
Aman Mehta and Anupam Datta
Context managers summarize or evict old messages so
agents can operate beyond a finite window. This creates a reliability problem: removing history that still shapes the model's computation may change later actions, while retaining everything defeats compression. We study plans because
agents create them early and are expected to use them later, yet behavioral success alone cannot reveal whether later calls still depend on the plan exchange. We make four contributions. First, we introduce replay pairing, which holds the action-observation trajectory fixed and compares pre-action hidden states with and without the plan exchange. We show that, on Llama-3.1-70B in ALFWorld, the mean contrast across 15 sampled layers decreases 4.1x from the first post-plan action to the second; Llama-3.1-8B shows a similar 5.6x decrease. Second, we show that a simple linear model using layer-32 hidden states predicts later contrast on entirely held-out tasks better than a baseline that only knows the action-call offset (R^2 = 0.847 vs. 0.575). Third, we identify confounds that limit this measurement: the paired histories differ in length,
token positions, and discourse structure, and replayed
reasoning can reintroduce plan-derived text.
Reasoning traces therefore require symmetric controls, and native Qwen3 shows that the Llama timing pattern is not universal. Fourth, in a separate descriptive compression test, we find pooled ALFWorld run success of 56.7% with full history and 22.0% with naive eviction; preserving or re-surfacing only the plan does not recover the gap. Replay pairing measures sensitivity to retaining an exchange, not isolated plan content or later eviction. It gives researchers a concrete audit tool and suggests that context policies should preserve broader task state, not plans alone.
Read the full abstract ▾
Computation and Language
Understanding Knowledge Distillation in Post-Training: When It Helps and When It Fails
Xin Liu, Simin Ma, Shujian Liu et al. (+5 more)
Large
language models (LLMs) achieve strong performance across many tasks, but their high computational cost limits deployment in resource-constrained environments. Knowledge
Distillation (KD) offers a practical solution by transferring knowledge from a teacher model of a larger size to a smaller student model. While prior work has mainly examined task-specific or small-scale settings, the post-training stage for building general instruction-following models has received limited
attention. In this paper, we conduct a systematic study of KD in post-training using the large-scale Tulu 3
dataset. We find that KD outperforms supervised fine-tuning (SFT) in low-data regimes, but its advantage diminishes as more training data is added. Distilling from a stronger instruction-tuned teacher restores substantial gains even with abundant data, indicating that KD remains effective when the teacher provides knowledge that the student cannot easily acquire from the training data alone. We further study domain-specific, low-resource scenarios and propose a two-stage KD strategy that leverages synthetic teacher-labeled data followed by refinement on human annotations. This method consistently improves student performance, providing practical guidance for building compact models in data-scarce environments.
Read the full abstract ▾
All authors Xin Liu, Simin Ma, Shujian Liu, Song Wang, Sathish Reddy Indurthi, Haoyun Deng, Lu Wang, Kaiqiang Song
Computation and Language · Cross-listed from cs.SD
Cross-lingual Retrieval-Augmented Classification for Dysarthria Severity Assessment
Taeyoung Jeong, Insung Lee, Du-Seong Chang et al. (+1 more)
Automatic dysarthria severity assessment is limited by the scarcity of labeled pathological
speech data. To address this, we propose Cross-lingual
Retrieval-Augmented Classification (CRAC), which leverages
speech from a different
language via an align-retrieve-fuse pipeline. Supervised contrastive learning first shapes a severity-focused
embedding space, then a vector database is built from the opposite-
language corpus. During both training and inference, the classifier retrieves top-k references from the aligned space and fuses them with the input via cross-
attention. Evaluated on Korean post-stroke and Italian ALS dysarthria
datasets under a speaker-independent three-class protocol, CRAC achieves balanced accuracies of 87.3% on Korean and 86.7% on Italian, improving over monolingual baselines by 8.4 and 20.0 percentage points, respectively.
Read the full abstract ▾
All authors Taeyoung Jeong, Insung Lee, Du-Seong Chang, Myoung-Wan Koo
Computation and Language
Explanation-Guided Medical Named Entity Recognition with Stability and Boundary Awareness for Atopic Dermatitis
Xueguang Li, Di Lin, Xue Jiang et al. (+2 more)
Objective: This study aims to improve the reliability and robustness of medical named entity recognition (NER) in Chinese atopic dermatitis (AD) clinical texts through explanation-guided learning. Methods: We propose a stability and boundary-aware explanation-guided NER framework. Perturbation-based analysis is used to evaluate explanation stability and entity boundary sensitivity. An adaptive fusion strategy dynamically combines local and global explanation to generate more reliable
token-level explanations. The fused explanation signals are further incorporated into model training through stability, boundary-aware, and consistency constraints. Results: Experiments on Chinese AD NER
datasets show that the proposed framework improves explanation robustness and achieves consistent performance gains across multiple NER models. The adaptive fusion strategy also provides more stable explanations and stronger boundary perception than individual explanation methods. Conclusion: The proposed method effectively integrates reliable explanation signals into medical NER training, improving both recognition performance and explanation reliability. The framework provides a practical and generalizable solution for explainable medical NER and offers reliable support for downstream clinical decision-making and medical knowledge applications.
Read the full abstract ▾
All authors Xueguang Li, Di Lin, Xue Jiang, Yanxi Li, Yugang Chi
Computation and Language
DynamicMem: A Long-Horizon Memory Benchmark in Real-World Settings
Wenya Xie, Shengming Zhou, Zelin Li et al. (+9 more)
LLM
agents increasingly act as personal assistants that must remember a user's profile over months: who they are (attributes), what they routinely do (habits), and what they prefer (preferences), and keep it updated as jobs, routines, and tastes drift. Existing
benchmarks evaluate this "
memory" ability through short, simplified interactions, missing three core properties of real behavior: the profile is heterogeneous, with attributes, habits, and preferences evolving on different timelines; changes are driven by external context such as seasons and life events; and evidence is rarely stated explicitly, instead scattered across many small actions in different apps that a
memory system must infer from. We introduce DynamicMem, a synthetic
benchmark that constructs 15 months of activity per user, providing long-term multi-app data that real users'
privacy keeps out of reach. It provides user-consistent trajectories averaging 2.2M
tokens and 1,772 grounded events per user across 16 applications such as e-commerce, fitness, and social platforms. The profile evolves over this period and is never given explicitly: each attribute, habit, or preference must be inferred from small signals scattered across apps. We evaluate at five quarterly checkpoints to track how systems scale as history grows. Benchmarking five representative systems exposes problems a single accuracy score hides: (i) profile reconstruction degrades with history length while service-task accuracy stays flat, despite both drawing on the same
memory; (ii) no system both keeps facts that stay true and replaces facts that change, with errors clustering on preferences and on naming the exact referent; and (iii) over 93% of failures trace to what the
memory retrieves, not to the model writing the answer, so the largest room for improvement lies in
memory itself. Code: https://wenyaxie023.github.io/DynamicMem/
Read the full abstract ▾
All authors Wenya Xie, Shengming Zhou, Zelin Li, Pouya Parsa, Shuang Zhou, Xinheng Ding, Chinmay Arvind, Guanchu Wang, Vladimir Braverman, Ali Payani, Yantao Zheng, Zirui Liu
Computation and Language · Cross-listed from cs.CV
SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning
SingGuard Team
Vision-
language models (VLMs) are increasingly deployed in consumer, medical, financial, and enterprise applications. This broad deployment expands the safety surface: risks can arise from
multimodal question answering, assistant responses, and cross-modal composition, while moderation policies may vary across products, regions, and deployment stages. Most existing guardrails either rely on fixed taxonomies or target only a narrow set of interaction settings, which limits their adaptability when safety rules change at deployment time. We present SingGuard, a policy-adaptive
multimodal guardrail model family for safety assessment in
multimodal conversations. SingGuard treats the active policy as a runtime input: given natural-
language rules, it checks the target content against the active policy rule by rule and predicts both the safety label and the triggered rule. To balance efficiency and interpretability, SingGuard supports fast, hybrid, and slow inference regimes along a fast-to-slow
reasoning spectrum, ranging from direct safety judgments to policy-grounded deliberation. We further optimize this behavior with fast–slow decoupled reinforcement learning. We also introduce SingGuard-Bench, a
multimodal guardrail
benchmark with 56{,}340 examples spanning 80+ fine-grained risk types across
multimodal QA,
adversarial attack, and dynamic-rule evaluation settings, including cross-modal joint-risk cases where each modality is harmless in isolation but their composition implies unsafe intent. Across six
benchmark families (35
datasets), SingGuard achieves state-of-the-art average F1 in every family. Dynamic-rule evaluation further shows improved policy-following accuracy from 0.6465 to 0.7415 under runtime policy shifts. Our code is available at https://github.com/inclusionAI/Sing-Guard.
Read the full abstract ▾
Computation and Language
Parth Bramhecha, Smit Deshmukh, Sairaj Bodhale et al. (+2 more)
As Large
Language Models (LLMs) achieve widespread integration across diverse linguistic landscapes, ensuring their safety and
alignment with regional normative values remains a critical challenge. Current safety mechanisms are predominantly optimized for English-centric frameworks, often failing to capture the unique socio-cultural sensitivities and localized categories of harm inherent to the Indic region. To address this gap, we introduce IndicGuard, a
multilingual safety guard model and
dataset for Indic
languages. We construct a high-volume, culturally nuanced safety
dataset encompassing ten major Indic
languages, systematically curated to capture regional harms, sensitive socio-political contexts, and
adversarial jailbreaks. Leveraging this corpus, we fine-tune a 4B-parameter instruction-tuned model based on Gemma-3-4B-IT to serve as a
multilingual safety guardrail for real-time content moderation and policy compliance checking. Our empirical evaluations demonstrate that IndicGuard significantly enhances LLM robustness against localized vulnerabilities, achieving high moderation consistency across different conversational turns. Crucially, IndicGuard consistently outperforms the existing baseline model, CultureGuard, across evaluated
languages. Finally, we demonstrate that our model effectively generalizes to low-resource Indic
languages excluded from training, substantiating the structural robustness and cross-lingual transfer capabilities of the framework.
Read the full abstract ▾
All authors Parth Bramhecha, Smit Deshmukh, Sairaj Bodhale, Adwait Borate, Raviraj Joshi
Computation and Language
Bagpiper-TTS: Natural Language Guided Universal Speech Synthesis
Jinchuan Tian, Haoran Wang, Siddhant Arora et al. (+6 more)
Classical TTS systems typically rely on rigid input formats and predefined metadata slots, limiting their ability to fulfill flexible user requirements. This paper introduces Bagpiper-TTS, a universal
speech synthesis system that deals with diverse natural
language user requests. Given a natural
language prompt, Bagpiper-TTS first reasons over the users' intent to derive a rich caption, i.e., a comprehensive textual blueprint encompassing both transcription and nuanced metadata. Subsequently, this caption guides the synthesis of the target
speech. Our model inherently supports a broad spectrum of tasks besides classical TTS applications, including multi-talker, intent-to-
speech, role-play synthesis, singing voice synthesis, and more. Experimental results demonstrate that Bagpiper-TTS achieves an 1.7% Word Error Rate (WER) on the Seed-TTS-Eval
benchmark and match the performance of dedicated models in both LLM-as-a-judge and human subjective evaluations across multiple applications.
Read the full abstract ▾
All authors Jinchuan Tian, Haoran Wang, Siddhant Arora, Takashi Maekaku, Keita Goto, Jin Sakuma, Yusuke Shinohara, Chao-Han Huck Yang, Shinji Watanabe
Computation and Language
Lychee-Reranker: Fast but Not Late Interaction for Compressed Document Reranking
Xinping Zhao, Jiaxin Xu, Ziqi Dai et al. (+9 more)
As
retrieval systems scale, effective and efficient reranking becomes increasingly important. However, most existing encoder- and decoder-based rerankers jointly process every query–passage pair, tightly coupling their online computation and limiting deployment efficiency and flexibility. We present Lychee-Reranker, a fast but not late-interaction (FBNL) reranker that decouples query and passage computation while retaining expressive relevance modeling. Built on an encoder–decoder architecture, Lychee-Reranker pre-encodes passages using Matryoshka
embedding pooling, while its decoder models system and user instructions together with query intent; cross-
attention then captures fine-grained relevance between the resulting query context and passage representations. Together, these designs offer four key advantages: (i) efficiency from offline passage encoding, (ii) expressiveness from cross-
attention, (iii) compactness from Matryoshka
embedding pooling, and (iv) test-time compute through an adjustable compute budget. We instantiate Lychee-Reranker in three sizes, Nano, Small, and Large, with 0.27B, 1B, and 4B activated parameters, respectively. Extensive experiments on BEIR, MIRACL, and LMEB demonstrate strong reranking performance with superior efficiency. On BEIR and MIRACL, our models achieve competitive performance in multi-domain and
multilingual reranking, on par with strong industrial rerankers such as the Qwen3/BGE-Reranker series. On LMEB-Dialogue, a compact
embedding model paired with our Nano reranker, which has only 0.27B activated parameters, remains competitive with 7–12B
embedding models. Lychee-Reranker was previously known as KaLM-Reranker-V1. Data and models are available at https://huggingface.co/collections/KaLM-
Embedding/lychee-kalm-reranker-and-jev. The code is available at https://github.com/HITsz-TMG/Lychee-Reranker.
Read the full abstract ▾
All authors Xinping Zhao, Jiaxin Xu, Ziqi Dai, Xin Zhang, Huiyao Chen, Shouzheng Huang, Xianhao Xiong, Danyu Tang, Xinshuo Hu, Guohong Fu, Meishan Zhang, Baotian Hu
Computation and Language
Does the Same Token Mean the Same State? MoE Routing as Signal for Reasoning Control
Kang Chen, Minshen Yu, Junjie Nian et al. (+3 more)
In sparse Mixture-of-Experts
language models, does the same
token id imply the same router state and the same experts producing it? Holding the emitted
token id fixed at repeated anchors, we find it does not: the experts that produce it still separate task context, trajectory history, and
reasoning-effort mode. This residual structure supports test-time control: near boundary anchors (the final-response transition) and delimiter anchors (which open the answer, e.g.\ \texttt{\textbackslash boxed\{} or code fences), routing neighborhoods already align with final-answer basins at a marker-only readout and strongest when the routing is read at the answer opening. We operationalize this as RAD (Routing Agreement Decoding), an answer-string-free multi-rollout selector: it locates a fixed anchor, represents each rollout by its anchor-window MoE routing states, and returns the densest Weighted-Jaccard $K$-NN route-basin center, without parsing, normalizing, executing, or voting over answer strings. Across 10 sparse-MoE configurations (gpt-oss, Qwen3-MoE) and 6
datasets spanning math, GPQA, and code, RAD is on par with Majority where string voting is well-posed, with small positive paired deltas (RAD $73.9$ / RAD+DC $74.2$ vs.\ Majority $73.6$). Like majority voting, RAD is not a verifier: a dense wrong basin can still win. Its value is the interface: the same selector gives direct pass@1 on code, where exact-string voting is ill-defined, and the same routing-density principle, re-anchored to the agentic boundary, improves best-of-16 patch selection on SWE-bench Verified over random, where patches have no answer string to vote on.
Read the full abstract ▾
All authors Kang Chen, Minshen Yu, Junjie Nian, Yaoning Wang, Yixin Cao, Yugang Jiang
Computation and Language · Cross-listed from cs.SI
Cross-National Information Attacks: A Two-Decade Analysis of Troll Behavior in Korea
Jaehong Kim, Hyeonseung Kim, Jiseon Kim et al. (+4 more)
Coordinated foreign influence operations pose a growing threat to online platforms, but detecting state-linked troll activity and tracking its
evolution remain challenging. This paper presents an explainable machine learning framework for theory-guided detection and longitudinal analysis of suspected trolling within Korean online news comment sections. Our hierarchical model classifies comments along three dimensions central to influence campaigns: foreign origin, moral-emotional framing, and target country. To support explainability, it also extracts brief span-level textual evidence that provides human-interpretable rationales. We apply the approach to 112M South Korean news comments authored by 4M users over nearly 20 years, identifying 23,998 accounts exhibiting behavior consistent with coordinated manipulation. Analyzing these accounts, we find that they predominantly rely on morally condemning rhetoric rather than direct promotion of foreign-aligned narratives; this rhetoric receives significantly higher user engagement. Among the highest-engagement comments, the moral condemnation most frequently targets domestic political figures (e.g., presidents or party leaders) on both the left and the right, potentially amplifying polarization. Our framework supports transparent platform governance through explainable, evidence-based moderation. These observed rhetorical and engagement patterns can inform how platforms and observatories prioritize defenses and intervene before harmful narrative-target combinations achieve widespread reach.
Read the full abstract ▾
All authors Jaehong Kim, Hyeonseung Kim, Jiseon Kim, Alice Oh, Thorsten Holz, Wonjae Lee, Meeyoung Cha
Computation and Language · Cross-listed from cs.IR
HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditions
Yuichi Tateno
With the rapid spread of
retrieval-augmented generation and semantic search, choosing the right
embedding and
retrieval configuration is increasingly hard. Large
retrieval benchmarks are comprehensive but too heavy to rerun during development, and there is little infrastructure for comparing production settings–dimensionality reduction, quantization, reranking–across many models under identical conditions. We present HAKARI-Bench, a lightweight
benchmark that reconstructs existing
retrieval suites into small
datasets (Nano-sets): 35
benchmarks and 551 tasks across 43
languages in a unified format, enabling same-condition, model-agnostic comparison of five
retrieval families (BM25, dense, sparse, late interaction, rerankers) and their efficiency variants. Across 55 models, its overall ranking reproduces the official MTEB
retrieval v2, MMTEB v2
retrieval, and English BEIR (full) at Spearman >0.97. HAKARI-Bench does not replace full evaluation; it enables rapid model selection, regression detection, and reading the quality-efficiency Pareto frontier. Code, data, and leaderboard are released under the MIT license.
Read the full abstract ▾
Computation and Language
Learning Moral Diversity: Modelling Individual Perspectives in Moral Classification of Texts
Yi Ren, Lewis Mitchell and Matthew Roughan
Understanding moral values in social media text offers insight into moral judgement formation, and supervised NLP models trained on crowdsourced data have achieved strong classification performance. However, most approaches simplify the problem by aggregating multiple annotators' labels into a single "ground truth", overlooking the inherent subjectivity of the task. In practice, there are disagreements between annotators caused by personal viewpoint or inherent ambiguities, particularly for short tweets. Here, we extend a pretrained
language model with a layer that learns annotator-specific features. Our model improves predictions of individual annotations and yields representations that reveal meaningful insights into annotators' moral perspectives. We show that models trained on aggregated labels may hide variation and give a misleading impression of performance. Overall, we demonstrate that disagreement reflects the inherent subjectivity of the task and that modelling individual perspectives creates benefits for moral classification of texts.
Read the full abstract ▾
Computation and Language
AI Fiction in the Wild
Neel Gupta, Maria Antoniak and Melanie Walsh
Some professional authors are beginning to use AI tools to help produce their fiction writing. Are readers using AI to generate fiction, too? Drawing on over 500,000 anonymized, English-
language ChatGPT-user conversations (arXiv:2405.01470), we find that more than one third of the conversations involve some form of fiction generation – including original stories, roleplay, fanfiction, and erotica. This AI-generated fiction is notably dominated by power users. We identify common fiction generation patterns and profiles among these users, including what we call "infinite story demanders," who repeatedly request and revise variations of the same or similar narratives over extended periods of time. We show that users especially gravitate toward fanfiction and erotica, and that they are broadly drawn to generic forms, repetition, immediacy, and niche combinations of story elements. Our findings motivate two theoretical provocations. First, we argue that AI technologies may lead to a shift in the conventional relationship between the author and reader, potentially producing what we call a "solipsistic reader-writer," who both generates and consumes fiction within a closed conversational loop, interacting with a machine rather than a human other. Second, we note that LLMs enable interactivity, play, and permutation in ways that are seemingly pleasurable for users, raising questions about where AI will fit into contemporary storytelling and entertainment ecosystems. We situate these developments within broader transformations in literature and media, including self-publishing, fanfiction, and pornography, and suggest that AI-generated fiction shares structural affinities with on-demand, personalized, and repetitive cultural forms.
Read the full abstract ▾
Computation and Language
Language-Specific Sentiment Polarity Biases in Encoder and Large Language Model Classification of Product Reviews
Advita Rajiv, Kavitha Kothur and Gautham Reddy
This study investigates sentiment polarity biases, specifically, differences in how accurately AI models classify positive versus negative reviews across
languages and model architectures. Large
language models show a negative bias in French and are more accurate on negative reviews, while encoder models exhibit positive bias in Japanese, missing negative reviews that use indirect criticism. These
language-specific polarity biases have implications in both social and business domains deploying
multilingual sentiment analysis systems.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
GroundEval: A Deterministic Replacement for LLM-as-Judge in Stateful Agent Evaluation
Jeffrey Flynt
Before letting an
agent operate over real context, can you prove it used the right evidence? GroundEval turns that question into a deterministic test of what the
agent searched, fetched, cited, and was permitted to access. In one case study, two frontier LLM judges scored a plausible
agent response 0.85 and higher. But the trace told a different story: the
agent had never retrieved the artifact its answer depended on, yielding a GroundEval score of 0.000. We introduce GroundEval, a judge-free framework for evaluating
agents against grounded, time-bounded, and access-controlled evidence. GroundEval uses a domain configuration to generate questions, lets the
agent choose how to answer, and then scores both the final answer and the recorded trajectory that produced it. The
benchmark targets three failures that LLM-as-judge evaluation struggles to detect: whether an
agent checked before claiming absence, reasoned only from evidence available to the actor at the relevant time, and used the correct causal mechanism rather than a plausible one. These correspond to three tracks: Silence, Perspective, and Counterfactual. GroundEval exposes when plausible answers rest on invalid evidence paths, and produces structured per-question diagnostics that pair tool activity with the
agent's turn-level narration, making each score inspectable rather than merely reported. Our case studies suggest this failure mode is common rather than exceptional, one that final-answer and judge-based evaluation cannot detect by construction.
Read the full abstract ▾