Computation and Language
Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarks
Wenya Xie, Qingying Xiao, Yu Zheng et al. (+8 more)
The rise of large
language models (LLMs) has transformed healthcare by offering clinical guidance, yet their direct deployment to patients poses safety risks due to limited domain expertise. To mitigate this, we propose repositioning LLMs as clinical assistants that collaborate with experienced physicians rather than interacting with patients directly. We conduct a two-stage inspiration-feedback survey to identify real-world needs in clinical workflows. Guided by this, we construct DoctorFLAN, a large-scale Chinese medical
dataset comprising 92,000 Q&A instances across 22 clinical tasks and 27 specialties. To evaluate model performance in doctor-facing applications, we introduce DoctorFLAN-test (550 single-turn Q&A items) and DotaBench (74 multi-turn conversations). Experimental results with over ten popular LLMs demonstrate that DoctorFLAN notably improves the performance of open-source LLMs in medical contexts, facilitating their
alignment with physician workflows and complementing existing patient-oriented models. This work contributes a valuable resource and framework for advancing doctor-centered medical LLM development
Read the full abstract ▾
All authors Wenya Xie, Qingying Xiao, Yu Zheng, Xidong Wang, Junying Chen, Ke Ji, Anningzhe Gao, Prayag Tiwari, Xiang Wan, Feng Jiang, Benyou Wang
Computation and Language
LogiNumSynth: Synthesizing Joint Logical-Numerical Reasoning Problems for Language Models
Yiwei Liu, Yucheng Li, Xiao Li et al. (+1 more)
Joint logical-numerical
reasoning remains a major challenge for
language models, yet existing
datasets rely on fixed rule sets and offer limited control over task complexity, constraining their generalizability for evaluation and training. We present LogiNumSynth, a flexible natural
language problem synthesizer that synthesizes tasks requiring proficiency in joint logical
reasoning (e.g., rule-based
reasoning) and numerical
reasoning (e.g., arithmetic computation). LogiNumSynth supports fine-grained control over
reasoning world richness, logical
reasoning depth, and the complexity of numerical computations, enabling flexible data synthesis across difficulty levels. We demonstrate three key contributions: (1) Synthesizer – synthesizing fully controllable joint
reasoning tasks over natural
language; (2) Evaluation & Process Analysis – evaluating both process accuracy and answer accuracy; (3) Targeted Training – using synthesized data to enhance LLMs'
reasoning performance. Experiments with multiple LLMs highlight persistent weaknesses in logical-numerical
reasoning, showing that LogiNumSynth can serve as both a diagnostic tool and a source of targeted supervision for advancing integrated
reasoning skills.
Read the full abstract ▾
All authors Yiwei Liu, Yucheng Li, Xiao Li, Gong Cheng
Computation and Language · Cross-listed from cs.MA
Automating Structural Engineering Workflows with Large Language Model Agents
Haoran Liang, Yufa Zhou, Mohammad Talebi Kalaleh et al. (+1 more)
We introduce $\textbf{MASSE}$, the first Multi-
Agent System for Structural Engineering, effectively integrating large
language model (LLM)-based
agents with real-world engineering workflows. Structural engineering is a fundamental yet traditionally stagnant domain, with core workflows remaining largely unchanged for decades despite its substantial economic impact and global
market size. Recent advancements in LLMs have significantly enhanced their ability to perform complex
reasoning, long-horizon planning, and precise tool utilization – capabilities well aligned with structural engineering tasks such as interpreting design codes, executing load calculations, and verifying structural capacities. We present a proof-of-concept showing that most real-world structural engineering workflows can be fully automated through a training-free LLM-based multi-
agent system. MASSE enables immediate deployment in professional environments, and our comprehensive validation on real-world case studies demonstrates that it can reduce expert workload from approximately two hours to mere minutes, while enhancing both reliability and accuracy in practical engineering scenarios.
Read the full abstract ▾
All authors Haoran Liang, Yufa Zhou, Mohammad Talebi Kalaleh, Qipei Mei
Computation and Language
DND: Boosting Large Language Models with Dynamic Nested Depth
Tieyuan Chen, Xiaodong Chen, Haoxing Chen et al. (+3 more)
We introduce Dynamic Nested Depth (DND), a novel method that improves performance for off-the-shelf LLMs by selecting critical
tokens to reprocess in a nested depth manner. Specifically, at the end of the given
transformer layer, DND identifies more critical
tokens with a router and feeds them back for an extra round of processing, effectively “reviewing" difficult
tokens while avoiding redundant computation for easier ones. The dynamic selection mechanism is tailored for precise control via two novel strategies: a router controlling loss to enhance
token selection distinguishability, and a threshold control scheme to ensure selection stability. We demonstrate the effectiveness of DND by directly integrating it into pre-trained dense and MoE models during a post-training phase. On diverse
benchmarks, this approach boosts the performances of the dense Qwen3-1.7B by 1.88% and the MoE Qwen3-30B-A3B by 0.87%, all with a minimal parameter and computing increase.
Read the full abstract ▾
All authors Tieyuan Chen, Xiaodong Chen, Haoxing Chen, Zhenzhong Lan, Weiyao Lin, Jianguo Li
Computation and Language
ABLEIST: Intersectional Disability Bias in LLM-Generated Hiring Scenarios
Mahika Phutane, Hayoung Jung, Matthew Kim et al. (+2 more)
Large
language models (LLMs) are increasingly under scrutiny for perpetuating identity-based discrimination in high-stakes domains such as hiring, particularly against people with disabilities (PwD). However, existing research remains largely Western-centric, overlooking how intersecting forms of marginalization–such as gender and caste–shape experiences of PwD in the Global South. We conduct a comprehensive audit of six LLMs across 2,820 hiring scenarios spanning diverse disability, gender, nationality, and caste profiles. To capture subtle intersectional harms and biases, we introduce ABLEIST (Ableism, Inspiration, Superhumanization, and Tokenism), a set of five ableism-specific and three intersectional harm metrics grounded in disability studies literature. Our results reveal significant increases in ABLEIST harms towards disabled candidates–harms that many state-of-the-art models failed to detect. These harms were further amplified by sharp increases in intersectional harms (e.g., Tokenism) for gender and caste-marginalized disabled candidates, highlighting critical blind spots in current safety tools and the need for intersectional safety evaluations of frontier models in high-stakes domains like hiring.
Read the full abstract ▾
All authors Mahika Phutane, Hayoung Jung, Matthew Kim, Tanushree Mitra, Aditya Vashistha
Computation and Language
Deep Research with Open-Domain Evaluation and Multi-Stage Guardrails for Safety
Wei-Chieh Huang, Henry Peng Zou, Yaozu Wu et al. (+12 more)
Deep research frameworks have shown promising capabilities in synthesizing comprehensive reports from web sources. While deep research possesses significant potential to address complex issues through planning and research cycles, existing frameworks are deficient in sufficient evaluation procedures and stage-specific protections. They typically treat evaluation as exact match accuracy of question-answering, but overlook crucial aspects of report quality such as credibility, coherence, breadth, depth, and safety. This oversight may result in hazardous or malicious sources being integrated into the final report. To address this, we introduce DeepResearchGuard, a framework featuring four-stage safeguards with open-domain evaluation, and DRSafeBench, a novel stage-wise safety
benchmark. Evaluating across GPT-4o, o4-mini, Gemini-2.5-flash, DeepSeek-v3, GPT-5, DeepResearchGuard improves defense success rates by 16.53% while reducing over-refusal to 6%. Through extensive experiments, we show that DRSafeBench enables comprehensive open-domain evaluation and stage-aware defenses that effectively block harmful content propagation, while systematically improving report quality without excessive over-refusal rates.
Read the full abstract ▾
All authors Wei-Chieh Huang, Henry Peng Zou, Yaozu Wu, Dongyuan Li, Yankai Chen, Weizhi Zhang, Yangning Li, Angelo Zangari, Jizhou Guo, Chunyu Miao, Liancheng Fang, Langzhou He, Yinghui Li, Renhe Jiang, Philip S. Yu
Computation and Language · Cross-listed from cs.CV
Huanjin Yao, Ruifei Zhang, Jiaxing Huang et al. (+8 more)
With the recent emergence of revolutionary autonomous agentic systems, research community is witnessing a significant shift from traditional static, passive, and domain-specific AI
agents toward more dynamic, proactive, and generalizable agentic AI. Motivated by the growing interest in agentic AI and its potential trajectory toward AGI, we present a comprehensive survey on Agentic
Multimodal Large
Language Models (Agentic MLLMs). In this survey, we explore the emerging paradigm of agentic MLLMs, delineating their conceptual foundations and distinguishing characteristics from conventional MLLM-based
agents. We establish a conceptual framework that organizes agentic MLLMs along three fundamental dimensions: (i) Agentic internal intelligence functions as the system's commander, enabling accurate long-horizon planning through
reasoning, reflection, and
memory; (ii) Agentic external tool invocation, whereby models proactively use various external tools to extend their problem-solving capabilities beyond their intrinsic knowledge; and (iii) Agentic environment interaction further situates models within virtual or physical environments, allowing them to take actions, adapt strategies, and sustain goal-directed behavior in dynamic real-world scenarios. To further accelerate research in this area for the community, we compile open-source training frameworks, training and evaluation
datasets for developing agentic MLLMs. Finally, we review the downstream applications of agentic MLLMs and outline future research directions for this rapidly evolving field. To continuously track developments in this rapidly evolving field, we will also actively update a public repository at https://github.com/HJYao00/Awesome-Agentic-MLLMs.
Read the full abstract ▾
All authors Huanjin Yao, Ruifei Zhang, Jiaxing Huang, Jingyi Zhang, Yibo Wang, Bo Fang, Ruolin Zhu, Yongcheng Jing, Shunyu Liu, Guanbin Li, Dacheng Tao
Computation and Language · Cross-listed from cs.CR
Secret-Protected Evolution for Differentially Private Synthetic Text Generation
Tianze Wang, Zhaoyu Chen, Jian Du et al. (+3 more)
Text data has become extremely valuable on large
language models (LLMs) and even lead to general artificial intelligence (AGI). A lot of high-quality text in the real world is private and cannot be freely used due to
privacy concerns. Therefore, differentially private (DP) synthetic text generation has been proposed, aiming to produce high-utility synthetic data while protecting sensitive information. However, existing DP synthetic text generation imposes uniform guarantees that often overprotect non-sensitive content, resulting in substantial utility loss and computational overhead. Therefore, we propose Secret-Protected
Evolution (SecPE), a novel framework that extends private
evolution with secret-aware protection. Theoretically, we show that SecPE satisfies $(\mathrm{p}, \mathrm{r})$-secret protection, constituting a relaxation of Gaussian DP that enables tighter utility-
privacy trade-offs, while also substantially reducing computational complexity relative to baseline methods. Empirically, across the OpenReview, PubMed, and Yelp
benchmarks, SecPE consistently achieves lower Fréchet Inception Distance (FID) and higher downstream task accuracy than GDP-based Aug-PE baselines, while requiring less noise to attain the same level of protection. Our results highlight that secret-aware guarantees can unlock more practical and effective
privacy-preserving synthetic text generation.
Read the full abstract ▾
All authors Tianze Wang, Zhaoyu Chen, Jian Du, Yingtai Xiao, Linjun Zhang, Qiang Yan
Computation and Language · Cross-listed from cs.AI
Revisiting Model Interpolation for Efficient Reasoning
Taiqiang Wu, Runming Yang, Tao Liu et al. (+2 more)
Model merging, typically on Instruct and Thinking models, has shown remarkable performance for efficient
reasoning. In this paper, we systematically revisit the simplest merging method that interpolates two weights directly. Particularly, we observe that model interpolation follows a three-stage evolutionary paradigm with distinct behaviors on the
reasoning trajectory. These dynamics provide a principled guide for navigating the performance-cost trade-off. Empirical results demonstrate that a strategically interpolated model surprisingly surpasses sophisticated model merging baselines on both efficiency and effectiveness. We further validate our findings with extensive ablation studies on model layers, modules, and decoding strategies. Ultimately, this work demystifies model interpolation and offers a practical framework for crafting models with precisely targeted
reasoning capabilities. Code is available at \href{https://github.com/wutaiqiang/MI}{Github}.
Read the full abstract ▾
All authors Taiqiang Wu, Runming Yang, Tao Liu, Jiahao Wang, Ngai Wong
Computation and Language
Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
Zhiwen Ruan, Yixia Li, He Zhu et al. (+4 more)
Large
language models (LLMs) primarily rely on supervised fine-tuning (SFT) as a key method to adapt pre-trained models to domain-specific tasks such as mathematical
reasoning. However, standard SFT uniformly penalizes all
tokens, neglecting that only a small subset of critical
tokens determines
reasoning correctness. This uniform supervision often causes reduced output diversity and limited generalization. We propose Critical
Token Fine-tuning (CFT), a simple yet effective approach that updates only
tokens identified as functionally indispensable via counterfactual perturbations. By focusing gradient signals on these decisive
reasoning steps while preserving the diversity of non-critical
tokens, CFT can enhance both generation and diversity. Extensive experiments on five models across three families (Qwen, OLMo, LLaMA) and eleven mathematical
reasoning benchmarks show that CFT, despite fine-tuning on less than 12% of
tokens, consistently outperforms standard SFT. Moreover, CFT enables test-time scaling through improved sampling diversity and provides a stronger initialization for reinforcement learning, sustaining performance gains in later training stages while maintaining higher
entropy for better exploration. These results highlight CFT as a practical and general framework for efficient and robust LLM fine-tuning.
Read the full abstract ▾
All authors Zhiwen Ruan, Yixia Li, He Zhu, Yun Chen, Peng Li, Yang Liu, Guanhua Chen
Computation and Language
RV-HATE: Reinforced Multi-Module Voting for Implicit Hate Speech Detection
Yejin Lee, Hyeseon Ahn and Yo-Sub Han
Hate
speech remains prevalent in human society and continues to evolve in its forms and expressions. Modern advancements in internet and online anonymity accelerate its rapid spread and complicate its detection. However, hate
speech datasets exhibit diverse characteristics primarily because they are constructed from different sources and platforms, each reflecting different linguistic styles and social contexts. Despite this diversity, prior studies on hate
speech detection often rely on fixed methodologies without adapting to data-specific features. We introduce RV-HATE, a detection framework designed to account for the
dataset-specific characteristics of each hate
speech dataset. RV-HATE consists of multiple specialized modules, where each module focuses on distinct linguistic or contextual features of hate
speech. The framework employs reinforcement learning to optimize weights that determine the contribution of each module for a given
dataset. A voting mechanism then aggregates the module outputs to produce the final decision. RV-HATE offers two primary advantages: (1)~it improves detection accuracy by tailoring the detection process to
dataset-specific attributes, and (2)~it also provides interpretable insights into the distinctive features of each
dataset. Consequently, our approach effectively addresses implicit hate
speech and achieves superior performance compared to conventional static methods. Our code is available at https://github.com/leeyejin1231/RV-HATE.
Read the full abstract ▾
Computation and Language
Judge Before Answer: Can MLLM Discern the False Premise in Question?
Jidong Li, Lingyong Fang, Haodong Zhao et al. (+2 more)
Multimodal large
language models (MLLMs) have witnessed astonishing advancements in recent years. Despite these successes, MLLMs remain vulnerable to flase premise problems. However, existing
benchmarks targeting this issue are limited in scope: they often lack fine-grained categorization, exhibit insufficient coverage, and thus fail to provide a rigorous evaluation of the ability of models to recognize false premises. To bridge this gap, we introduce a fully automated pipeline for constructing a comprehensive
benchmark of false premise questions. Our method systematically categorizes the premises into three main types and thirteen subtypes according to the abilities required to identify the premises, resulting in the JBA
dataset.Results show current MLLMs still struggle with false premise recognition. Building upon this
benchmark, we further propose a recognition enhancement framework tailored to strengthen the robustness of MLLMs to detect false premises. Extensive experiments demonstrate that models trained with our framework achieve significant improvements in false premise recognition.
Read the full abstract ▾
All authors Jidong Li, Lingyong Fang, Haodong Zhao, Sufeng Duan, Gongshen Liu
Computation and Language
Obfuscation Rules for Detecting and Detoxifying Korean Toxicity
Yejin Lee, Su-Hyeon Kim, Hyundong Jin et al. (+3 more)
As
language models become increasingly deployed in online environments, toxicity detection and detoxification have received growing
attention. Existing studies primarily focus on non-obfuscated text, which limits robustness when users intentionally disguise toxic expressions. In particular, Korean toxic expressions can be easily disguised through agglutinative morphology and Hangeul-specific orthographic variation. However, obfuscation in Korean remains largely unexplored, which motivates us to introduce a KOTOX: Korean toxic
dataset for deobfuscation and detoxification. We categorize Korean obfuscation patterns into linguistically grounded classes, define transformation rules derived from real-world examples, and provide the resulting obfuscation framework as an open transformation package. Using these rules, we provide paired neutral and toxic sentences alongside their obfuscated counterparts. Models trained on our
dataset better handle obfuscated text without sacrificing performance on non-obfuscated text. This is the first
dataset that simultaneously supports deobfuscation and detoxification for the Korean
language. We expect the
dataset to facilitate better understanding and mitigation of obfuscated toxic content in LLM for Korean. Our code and data are available at https://github.com/leeyejin1231/KOTOX.
Read the full abstract ▾
All authors Yejin Lee, Su-Hyeon Kim, Hyundong Jin, Dayoung Kim, Yeonsoo Kim, Yo-Sub Han
Computation and Language · Cross-listed from cs.LG
Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning
Xiaoyun Zhang, Xiaojian Yuan, Di Huang et al. (+6 more)
Reasoning ability has become a defining capability of Large
Language Models (LLMs), with Reinforcement Learning with Verifiable Rewards (RLVR) emerging as a key paradigm to enhance it. However, RLVR training often suffers from policy
entropy collapse, where the policy becomes overly deterministic, hindering exploration and limiting
reasoning performance. While
entropy regularization is a common remedy, its effectiveness is highly sensitive to the fixed coefficient, making it unstable across tasks and models. In this work, we revisit
entropy regularization in RLVR and argue that its potential has been largely underestimated. Our analysis shows that (i) tasks of varying difficulty demand distinct exploration intensities, and (ii) balanced exploration may require the policy
entropy to be maintained within a moderate range below its initial level. Therefore, we propose Adaptive
Entropy Regularization (AER)–a framework that dynamically balances exploration and exploitation via three components: difficulty-aware coefficient allocation, initial-anchored target
entropy, and dynamic global coefficient adjustment. Experiments on multiple mathematical
reasoning benchmarks show that AER consistently outperforms baselines, improving both
reasoning accuracy and exploration capability.
Read the full abstract ▾
All authors Xiaoyun Zhang, Xiaojian Yuan, Di Huang, Wang You, Chen Hu, Jingqing Ruan, Ai Jian, Kejiang Chen, Xing Hu
Computation and Language
Punctuation-aware treebank tree binarization
Eitan Klinger, Vivaan Wadhwa and Jungyeul Park
This article presents a curated resource and evaluation suite for punctuation-aware treebank binarization. Standard binarization pipelines drop punctuation before head selection, which alters constituent shape and harms head-child identification. We release (1) a reproducible pipeline that preserves punctuation as sibling nodes prior to binarization, (2) derived artifacts and metadata (intermediate @X markers, reversibility signatures,
alignment indices), and (3) an accompanying evaluation suite covering head-child prediction, round-trip reversibility, and structural compatibility with derivational resources (CCGbank). On the Penn Treebank, punctuation-aware preprocessing improves head prediction accuracy from 73.66% (Collins rules) and 86.66% (MLP) to 91.85% with the same classifier, and achieves competitive
alignment against CCGbank derivations. All code, configuration files, and documentation are released to enable replication and extension to other corpora.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.MA
The Social Cost of Intelligence: Emergence, Propagation, and Amplification of Stereotypical Bias in Multi-Agent Systems
Thi-Nhung Nguyen, Linhao Luo, Amardeep Kaur et al. (+5 more)
Bias in large
language models (LLMs) remains a persistent challenge, often leading to stereotyping and unfair treatment across social groups. While prior work has mainly focused on individual LLMs, the emergence of multi-
agent systems (MAS), where multiple LLMs collaborate and communicate, introduces new and underexplored dynamics in how bias emerges, propagates, and amplifies. To systematically investigate these dynamics, we propose a simple evaluation framework with three
agent-level metrics that quantify bias emergence, propagation, and amplification throughout multi-
agent interaction. We evaluate MAS across three bias
benchmarks under varying LLM backbones, social-group configurations, communication behaviors, and
adversarial settings. Our results show that communication can trigger up to 70% new bias emergence, propagate bias across over 80% of
agents, and amplify stereotypes by more than 3$\times$. We further find that denser and competitive communication generally increases bias. Finally, we demonstrate that MAS are highly vulnerable to simple bias injection attacks, and existing defense strategies provide only limited protection. Our findings provide important insights into the
fairness and robustness of multi-
agent LLM systems.
Read the full abstract ▾
All authors Thi-Nhung Nguyen, Linhao Luo, Amardeep Kaur, Rollin Omari, Tamas Abraham, Junae Kim, Thuy-Trang Vu, Dinh Phung
Computation and Language
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF: A Reproducibility Study
Anirudh Ganesh and Jayavardhan Reddy
We present a reproducibility study of the state-of-the-art neural architecture for sequence labeling proposed by Ma and Hovy (2016)\cite{ma2016end}. The original BiLSTM-CNN-CRF model combines character-level representations via Convolutional Neural Networks (CNNs), word-level context modeling through Bi-directional Long Short-Term
Memory networks (BiLSTMs), and structured prediction using Conditional Random Fields (CRFs). This end-to-end approach eliminates the need for hand-crafted features while achieving excellent performance on named entity recognition (NER) and part-of-
speech (POS) tagging tasks. Our implementation successfully reproduces the key results, achieving 91.18% F1-score on CoNLL-2003 NER and demonstrating the model's effectiveness across sequence labeling tasks. We provide a detailed analysis of the architecture components and release an open-source PyTorch implementation to facilitate further research.
Read the full abstract ▾
Computation and Language
Evaluating Language Models' Evaluations of Games
Katherine M. Collins, Cedegao E. Zhang, Graham Todd et al. (+9 more)
Reasoning is not just about solving problems – it is also about evaluating which problems are worth solving at all. Evaluations of artificial intelligence (AI) systems primarily focused on problem solving, historically by studying how models play games such as chess and Go. In this paper, we advocate for a new paradigm that assesses AI systems' evaluation of games. First, we introduce a formalism for evaluating such evaluations. We then leverage a large-scale
dataset of over 100 novel board games and over 450 human judgments to compare evaluations produced by modern
language and
reasoning models against those of people and symbolic computational
agents. We consider two kinds of evaluative queries: assessing the payoff (or
fairness) and the funness of games. These queries span two dimensions relevant to the design of evaluations of AI evaluations: how complex a query is to compute and how difficult a query is to quantify. Our results show that
reasoning models are generally more aligned to people in their evaluations of games than non-
reasoning language models. However, we observe a non-monotonic relationship: as models get closer to game-theoretic optimal, their fit to human data weakens. We also observe more "jaggedness" across models for assessing funness, in line with the greater difficulty of quantifying this query. Across queries and games,
reasoning models show highly variable and unpredictable resource usage when assessing queries, pointing to the importance of imbuing more resource-rational meta-
reasoning in
language and
reasoning models.
Read the full abstract ▾
All authors Katherine M. Collins, Cedegao E. Zhang, Graham Todd, Lance Ying, Mauricio Barba da Costa, Ryan Liu, Prafull Sharma, Adrian Weller, Ionatan Kuperwajs, Lionel Wong, Joshua B. Tenenbaum, Thomas L. Griffiths
Computation and Language
GapDNER: A Gap-Aware Grid Tagging Model for Discontinuous Named Entity Recognition
Yawen Yang, Fukun Ma, Shiao Meng et al. (+2 more)
In biomedical fields, one named entity may consist of a series of non-adjacent
tokens and overlap with other entities. Previous methods recognize discontinuous entities by connecting entity fragments or internal
tokens, which face challenges of error propagation and decoding ambiguity due to the wide variety of span or word combinations. To address these issues, we deeply explore discontinuous entity structures and propose an effective Gap-aware grid tagging model for Discontinuous Named Entity Recognition, named GapDNER. Our GapDNER innovatively applies representation learning on the context gaps between entity fragments to resolve decoding ambiguity and enhance discontinuous NER performance. Specifically, we treat the context gap as an additional type of span and convert span classification into a
token-pair grid tagging task. Subsequently, we design two interactive components to comprehensively model
token-pair grid features from both intra- and inter-span perspectives. The intra-span regularity extraction module employs the biaffine mechanism along with linear
attention to capture the internal regularity of each span, while the inter-span relation enhancement module utilizes criss-cross
attention to obtain semantic relations among different spans. At the inference stage of entity decoding, we assign a directed edge to each entity fragment and context gap, then use the BFS
algorithm to search for all valid paths from the head to tail of grids with entity tags. Experimental results on three
datasets demonstrate that our GapDNER achieves new state-of-the-art performance on discontinuous NER and exhibits remarkable advantages in recognizing complex entity structures.
Read the full abstract ▾
All authors Yawen Yang, Fukun Ma, Shiao Meng, Aiwei Liu, Lijie Wen
Computation and Language · Cross-listed from cs.LG
Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation
Hengyuan Zhang, Shiping Yang, Xiao Liang et al. (+8 more)
Training student models on synthetic data generated by strong teacher models is a promising way to distilling the capabilities of teachers. However, recent studies show that stronger models are not always optimal teachers, revealing a mismatch between teacher outputs and student learnability. To address this issue, we propose PerSyn (Personalized data Synthesis), a novel synthesis strategy that operates under a new “Route then Generate” paradigm to create data tailored to each student model, enabling it to learn more effectively. Specifically, PerSyn first assigns each
prompt to its optimal teacher via a query-level router that jointly considers student learnability and teacher response quality. Each teacher then synthesizes data only for its assigned
prompts, making the process more efficient than the conventional “Generate then Select” paradigm, where all teachers must generate parallel responses for the entire
prompt set before constructing the final
dataset. Extensive experiments across different model families and scales demonstrate that PerSyn consistently achieves superior or comparable performance to all baselines in instruct tuning and math
reasoning settings. Further analysis verifies the effectiveness of PerSyn and offers extra insights to propel future research.
Read the full abstract ▾
All authors Hengyuan Zhang, Shiping Yang, Xiao Liang, Chenming Shang, Yuxuan Jiang, Chaofan Tao, Jing Xiong, Hayden Kwok-Hay So, Ruobing Xie, Angel X. Chang, Ngai Wong
Computation and Language
ADVICE: Answer-Dependent Verbalized Confidence Estimation
Ki Jung Seo, Sehun Lim and Taeuk Kim
Recent progress in large
language models (LLMs) has enabled them to communicate their confidence in natural
language, improving transparency and reliability. However, this expressiveness is often accompanied by systematic overconfidence, whose underlying causes remain poorly understood. In this work, we analyze the dynamics of verbalized confidence estimation and identify answer-independence – the failure to condition confidence on the model's own answer – as a primary driver of this behavior. To address this, we introduce ADVICE (Answer-Dependent Verbalized Confidence Estimation), a fine-tuning framework that promotes answer-grounded confidence estimation. Extensive experiments show that ADVICE substantially improves confidence calibration, while exhibiting strong generalization to unseen settings without degrading task performance. We further demonstrate that these gains stem from enhanced answer dependence, shedding light on the origins of overconfidence and enabling trustworthy confidence verbalization.
Read the full abstract ▾
Computation and Language
LLM$\times$MapReduce-V3: Enabling Interactive In-Depth Survey Generation through a MCP-Driven Hierarchically Modular Agent System
Yu Chao, Siyu Lin, xiaorong wang et al. (+7 more)
We introduce LLM x MapReduce-V3, a hierarchically modular
agent system designed for long-form survey generation. Building on the prior work, LLM x MapReduce-V2, this version incorporates a multi-
agent architecture where individual functional components, such as skeleton initialization, digest construction, and skeleton refinement, are implemented as independent model-context-protocol (MCP) servers. These atomic servers can be aggregated into higher-level servers, creating a hierarchically structured system. A high-level planner
agent dynamically orchestrates the workflow by selecting appropriate modules based on their MCP tool descriptions and the execution history. This modular decomposition facilitates human-in-the-loop intervention, affording users greater control and customization over the research process. Through a multi-turn interaction, the system precisely captures the intended research perspectives to generate a comprehensive skeleton, which is then developed into an in-depth survey. Human evaluations demonstrate that our system surpasses representative baselines in both content depth and length, highlighting the strength of MCP-based modular planning.
Read the full abstract ▾
All authors Yu Chao, Siyu Lin, xiaorong wang, Zhu Zhang, Zihan Zhou, Haoyu Wang, Shuo Wang, Jie Zhou, Zhiyuan Liu, Maosong Sun
Computation and Language
Rethinking Agentic Workflows: Evaluating Inference-Based Test-Time Scaling Strategies in Text2SQL Tasks
Jiajing Guo, Kenil Patel, Jorge Piazentin Ono et al. (+2 more)
Large
language models (LLMs) are increasingly powering Text-to-SQL (Text2SQL) systems, enabling non-expert users to query industrial databases using natural
language. While test-time scaling strategies have shown promise in LLM-based solutions, their effectiveness in real-world applications, especially with the latest
reasoning models, remains uncertain. In this work, we
benchmark six lightweight, industry-oriented test-time scaling strategies and four LLMs, including two
reasoning models, evaluating their performance on the BIRD Mini-Dev
benchmark. Beyond standard accuracy metrics, we also report inference latency and
token consumption, providing insights relevant for practical system deployment. Our findings reveal that Divide-and-Conquer prompting and few-shot demonstrations consistently enhance performance for both general-purpose and
reasoning-focused LLMs. However, introducing additional workflow steps yields mixed results, and base model selection plays a critical role. This work sheds light on the practical trade-offs between accuracy, efficiency, and complexity when deploying Text2SQL systems.
Read the full abstract ▾
All authors Jiajing Guo, Kenil Patel, Jorge Piazentin Ono, Wenbin He, Liu Ren