Computation and Language
An Empirical Analysis of Factual Errors in Human-Written Text and Its Application to Factual Error Detection
Kazuma Iwamoto, Kazumasa Omura and Shotaro Ishihara
Factual Error Detection (FED), which is the task of identifying factually incorrect spans in a given text, has long been recognized as an important research problem. However, with the rapid rise of large
language models (LLMs), research
attention has shifted toward factual errors specific to LLM-generated text (
hallucinations) and their detection. As a result, the detection of factual errors in human-written text has been relatively neglected. To address this gap, we first distill a taxonomy of human-induced factual errors by analyzing corrections of newspaper articles, a representative source of text that is guaranteed to be human-written and contains few grammatical errors. Our analysis revealed that there are characteristic categories such as kanji misconversions and unit errors, which are not focused in existing
hallucination benchmarks. Based on the taxonomy, we then evaluate the FED capability of vanilla LLMs on synthesized realistic test cases and real corrections. Experimental results demonstrated that even high-performance LLMs such as GPT-5.4 achieved only word-level F1 score of 52% on the synthetic evaluation data, highlighting the task difficulty. Furthermore, a detailed analysis by detection difficulty revealed the current state of FED.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CY
AI Persuasive Framing in Collective Dilemmas
Anders Giovanni Møller, Alessia Galdeman, Arianna Pera et al. (+1 more)
AI
agents are promising tools that can act as flexible behavioral nudges to enhance human cooperation in addressing large-scale societal problems. However, evidence on whether AI
agents can effectively boost cooperation remains mixed. We recruited 1,283 participants to play iterated Collective Risk Games in small groups, testing whether AI assistants could nudge participants toward cooperation. By using persuasive framing personalized to each player's Social Value Orientation profile, the AI interventions significantly increased contributions and group success rates. These cooperative effects were short-lived, however, fading after the first few rounds. Strikingly, when the AI treatments were reconfigured to promote selfish behavior through exculpatory framing, the negative effects on contributions and group success were larger and substantially more persistent, particularly for personalized interventions. This asymmetry between prosocial and antisocial persuasion highlights the dual-use risks of AI systems designed to influence group behavior in collective action settings.
Read the full abstract ▾
All authors Anders Giovanni Møller, Alessia Galdeman, Arianna Pera, Luca Maria Aiello
Computation and Language
VASAE: Naming SAE Dictionary Directions with Vocabulary-Aligned Anchoring
Kairui Zhang, Ziwen Yu, Zahraa S. Abdallah et al. (+1 more)
Sparse autoencoders (SAEs) provide useful decompositions of
Transformer residual streams, but their learned features are usually named post hoc rather than directly connected to the
Transformer's
token vocabulary. We introduce Vocabulary-Aligned Sparse Autoencoder (VASAE), a method that trains SAE features under vocabulary-aligned anchoring and assigns each feature an intrinsic
token name: the
token string whose
embedding is nearest to that feature. Without reducing reconstruction quality compared with a standard SAE, VASAE produces dictionaries with vocabulary-aligned features. Using a 0.8 cutoff on the nearest-
token alignment score, dictionaries trained on GPT-2-small post-residual streams align about 90% of features in layers 0–10. In Llama-3.1-8B, representative shallow and middle-layer dictionaries contain strongly aligned features, including 92.8% in the shallow layer, while the representative final-layer dictionary shows limited
alignment. After subtracting the sentence-level mean sparse code, case studies show that many remaining intrinsic
token names are relevant to nearby input
tokens. These results suggest that vocabulary-aligned anchoring can connect learned features to intrinsic
token names during training, complementing post hoc interpretation of learned dictionaries.
Read the full abstract ▾
All authors Kairui Zhang, Ziwen Yu, Zahraa S. Abdallah, Martha Lewis
Computation and Language · Cross-listed from cs.AI
Verifiable Geometry Problem Solving: Solver-Driven Autoformalization and Theorem Proposing
Can Li, Ting Zhang, Junbo Zhao et al. (+1 more)
Geometry Problem Solving have increasingly adopt the neuro-symbolic paradigm, combining neural intuition with symbolic rigor. However, current frameworks suffer from severe bottlenecks in two core stages: autoformalization, which treats
multimodal translation as a static task decoupled from downstream solver compatibility, and
theorem prediction, where solvers frequently hit a deductive impasse due to fixed rule libraries. To address these, we propose SD-GPS, a solver-driven framework that treats the symbolic solver as an execution oracle throughout both formalization and deduction. First, Solver-Driven Autoformalization unifies supervised formal-
language adaptation and solvability-guided reinforcement learning into a single module built on QwenVL3-2B, making executability the central training signal. Second, Verified
Theorem Proposing introduces an impasse-aware
agent that proposes local auxiliary lemmas from current proof states, ensuring soundness by filtering all proposals through symbolic verification. Empirical evaluations on Geometry3K and PGPS9K demonstrate that SD-GPS consistently outperforms existing MLLM, neural, and neuro-symbolic methods across standard completion, multiple-choice, and cross-modal reference regimes, proving that closing the loop between
multimodal perception and symbolic execution significantly improves
geometric reasoning, offering profound insights into how neural
agents can be grounded by formal systems to achieve verifiable problem-solving capabilities.
Read the full abstract ▾
All authors Can Li, Ting Zhang, Junbo Zhao, Hua Huang
Computation and Language
Triadic Werewolf: A Jester Role for Multi-Hop Theory of Mind in LLMs
Avni Mittal
Theory-of-mind evaluations of large
language models typically use dyadic social-deduction games, where every observable cue points to a single hidden side, so a model with strong
language priors can score well without ever simulating opponents' incentives. We extend the Werewolf game with a Jester, a third faction whose utility on peer suspicion is inverted because it wins by being voted out, so optimal play requires
reasoning across three opposing utility functions. Across 60 games on GPT-4.1, DeepSeek-V3.1, and Llama-3.3-70B with Jester self-learning on and off, the Jester wins 60-70% of games while Werewolves never exceed 20%, and GPT-4.1 wolves vote the Jester out on day 1 in 60-70% of games, a strictly self-defeating action. Self-learning helps DeepSeek and Llama but hurts GPT-4.1, with the cost landing on Villagers rather than Werewolves. Only DeepSeek learns the subtle strategy of looking suspicious without looking intentionally suspicious, and it gains the most from the loop. Triadic incentive structure exposes a layer of multi-
agent reasoning that dyadic deduction games leave invisible.
Read the full abstract ▾
Computation and Language
A Study of Temporal Fusion Strategies for Named Entity Recognition in Historical Texts
Emanuela Boros
Temporal variation poses a unique challenge for named entity recognition (NER) in historical texts, where entities drift in surface form and salience across time. While
language models (LMs) have made progress in various NLP tasks, their ability to reason about temporality, especially in diachronic contexts, remains limited or at least, questionable. In this paper, we systematically study how temporal metadata can be structurally embedded into NER models using a range of lightweight fusion strategies. We experiment with both absolute and relative temporal representations, injected into
Transformer-based architectures via early or late fusion mechanisms such as cross-
attention, adapters, and concatenation. Our evaluations on French and German historical
datasets reveal that late fusion strategies yield more robust and temporally generalisable performance, particularly in early and noisy periods.
Read the full abstract ▾
Computation and Language
Learning Complementary Action Modeling from Automotive Maintenance Instructions
Jiaqi Wu, Bai Li, Jochen Hartmann et al. (+2 more)
A minute lexical variation can reverse the procedural meaning of an instruction even when the rest of the sentence remains unchanged. In automotive maintenance instructions, this pattern often appears when an action phrase turns an instruction into its procedural counterpart. The entities, modifiers, and surrounding context remain largely invariant, while the action phrase determines the procedural relation. We define this task as Complementary Action Modeling (CAM). Given a maintenance instruction, the goal is to identify or generate its procedural counterpart by modifying the action phrase while preserving the remaining sentence context. This task focuses on three aspects: distinguishing complementarity from surface similarity, controlling generation at the action-phrase level, and evaluating relational correctness using
retrieval, overlap-based, and human evaluation. Using a German automotive maintenance
dataset, we examine these questions through candidate matching and controlled Seq2Seq generation. The results show that complementary maintenance instructions are best modeled as procedural associations grounded in subtle lexical cues. They should therefore not be treated as ordinary cases of sentence similarity or synonym-based paraphrasing.
Read the full abstract ▾
All authors Jiaqi Wu, Bai Li, Jochen Hartmann, Martin Gaedke, Sander Stuijk
Computation and Language
Position Bias Correction is Insufficient for One-Pass Attention Sorting
Qiong Tang, Xiangkun Hu, Xiangyang Liu et al. (+2 more)
Long-context
language models suffer from position bias, where information in middle positions is underutilized.
Attention Sorting addresses this by iteratively reordering documents based on
attention patterns, but its multiple sort-and-generate cycles increase deployment cost. We hypothesize that position bias is the primary bottleneck and propose Debiased One-Pass
Attention Sorting, which estimates a per-
prompt position-bias curve from the low-
attention majority of documents and uses it to correct raw
attention scores (via subtraction or division) to enable single-pass sorting. Our experiments on two models refute this hypothesis in the tested setting: on LLaMA-2-7B-32K-Instruct, debiasing produces identical results to uncalibrated single-pass sorting (94.83% containment accuracy), while on YaRN-Llama-2-7b-64k, debiasing improves accuracy by 8.67 percentage points but remains 14.84pp behind iterative sorting, closing only 37% of the gap. These results suggest that position-bias correction is insufficient to match iterative sorting, and that repeated reordering provides additional benefits beyond bias correction.
Read the full abstract ▾
All authors Qiong Tang, Xiangkun Hu, Xiangyang Liu, Yiran Chen, Yunfan Shao
Computation and Language
NLL-Guided Full-Attention Layer Selection for Training-Free Sliding-Window Adaptation
Qiong Tang, Xiangkun Hu, Xiangyang Liu et al. (+2 more)
Hybrid
attention models that mix full and sliding-window
attention across layers offer a promising approach to efficient long-context inference, but the critical question of which layers should retain full
attention remains unsolved. Existing methods use either fixed periodic patterns or
attention-based heuristics that may not capture what matters for downstream accuracy. We propose NLL-guided layer selection, a training-free method that directly measures each layer's importance by computing the negative log-likelihood degradation on answer
tokens when that layer uses sliding-window instead of full
attention. On LongMemEval with Qwen3-4B, our method achieves 64.6% accuracy using only 1/4 full-
attention layers, matching the 1/2-FA periodic baseline (65.0%) while halving the computational budget. NLL-guided selection outperforms the SWAA-reported periodic 1/4-FA baseline by 10.4 percentage points and a matched LightTransfer-style baseline by 26.4 percentage points. De-confounding analysis shows the signal is consistent with long-range
attention needs rather than generic layer sensitivity. The method requires only $\sim$15 minutes of one-time calibration, advancing the efficiency-accuracy Pareto frontier for long-context LLM deployment.
Read the full abstract ▾
All authors Qiong Tang, Xiangkun Hu, Xiangyang Liu, Yiran Chen, Yunfan Shao
Computation and Language
SPARK: Adaptive Parametric Knowledge Participation for Faithful Retrieval-Augmented Generation
Ruochang Li, Pengcheng Huang, Zhenghao Liu et al. (+5 more)
Retrieval-augmented generation (RAG) enhances the faithfulness of large
language models (LLMs) by incorporating external knowledge into the generation process. However, conflicts between retrieved context and parametric knowledge can cause LLMs to rely on inconsistent internal knowledge, hindering their ability to faithfully leverage retrieved evidence. Existing approaches mitigate this issue by editing parametric knowledge to resolve such conflicts. However, locating and modifying knowledge encoded in model parameters is challenging and may inadvertently compromise the backbone LLM's capabilities. In this paper, we introduce SPARK, a novel framework that formulates parametric knowledge control as a latent participation policy learning problem, enabling LLMs to adaptively regulate the contribution of parametric knowledge during generation. Specifically, SPARK introduces a lightweight participation policy that dynamically modulates the contribution of FFN-mediated parametric knowledge while keeping the backbone LLM frozen. The participation policy is optimized with a faithfulness-oriented reward, enabling fine-grained control over parametric knowledge participation during generation. Extensive experiments on six
datasets validate the effectiveness of SPARK, which introduces and optimizes only 0.01% additional trainable parameters while keeping the backbone LLM fully frozen, thereby preserving its general capabilities and achieving more faithful generation than competitive baselines. All
datasets and code are available at https://github.com/OpenBMB/SHIFT.
Read the full abstract ▾
All authors Ruochang Li, Pengcheng Huang, Zhenghao Liu, Yukun Yan, Huiyuan Xie, Yu Gu, Ge Yu, Maosong Sun
Computation and Language
Output-Space Allocation Costs for Calibration-Guided LLM Compression: An Empirical Study
Qiong Tang, Xiangkun Hu, Xiangyang Liu et al. (+2 more)
Training-free compression methods for large
language models (LLMs) often use calibration data to guide compression decisions. ROCKET, a recent method combining sparse-dictionary factorization with multi-choice knapsack problem (MCKP) allocation, derives its per-layer factorization from an output reconstruction objective but uses weight-space Frobenius error as the MCKP allocation cost. We investigate whether aligning the allocation cost with the output-space objective improves compressed model fidelity. On Qwen3-8B at 50% compression, our ROCKET-ActCost achieves +0.8 percentage points higher average accuracy across 8 zero-shot
benchmarks (53.1% vs 52.3%), but increases WikiText perplexity by 16% (61.46 vs 52.98). This accuracy-perplexity tradeoff reveals that different allocation objectives favor different downstream metrics. The high correlation ($>$0.99) between weight-space and output-space errors limits allocation divergence, explaining the modest effect size. On Llama-3.2-1B at 20% compression, the two methods produce near-identical results (53.3% vs 53.5% accuracy, 14.45 vs 14.66 PPL), suggesting that the effect of the cost function is minor at lower compression ratios.
Read the full abstract ▾
All authors Qiong Tang, Xiangkun Hu, Xiangyang Liu, Yiran Chen, Yunfan Shao
Computation and Language
KG2Cypher: Data-Centric Pipeline for Building Enterprise Text-to-Cypher Systems
Minjun Choi, Yerin Kim, Junghyuk Seo et al. (+3 more)
Enterprise Knowledge Graphs (KGs) are increasingly used for internal search, analytics, and question answering, but building natural-
language interfaces for private enterprise graphs remains costly. We present KG2Cypher, a data-centric pipeline for building enterprise text-to-Cypher systems from existing KGs. KG2Cypher first constructs an executable Cypher query from observed graph facts and then uses LLMs to generate its associated natural-
language question. The resulting Text-Cypher pairs are validated with an LLM judge and human validation, and are converted into candidate-aware SFT data. The trained generator is served with class-conditioned schema prompting, entity
retrieval, and LoRA-based inference. We evaluate KG2Cypher in Korean enterprise settings, where short search-style queries and schema paraphrases make
language grounding difficult. LoRA SFT improves execution-result F1 from 0.806 to 0.950 on broadcast-program queries and from 0.70 to 0.92 on company queries. In an 11-class setting, KG2Cypher achieves 95.2% exact match, 99.9% execution rate, and 0.964 execution-result F1.
Read the full abstract ▾
All authors Minjun Choi, Yerin Kim, Junghyuk Seo, Sujin Mo, Hyemin Lee, Youngjoong Ko
Computation and Language
Enhancing Numerical Prediction in LLMs via Smooth MMD Alignment
Zhuo Zuo, Li Yue, Wenhao Zheng et al. (+2 more)
Despite their strong general capabilities, large
language models (LLMs) often remain unreliable when outputs must be numerically precise. A key reason is the training objective: standard cross-
entropy treats numeric
tokens as unstructured categories and ignores the metric structure of their values. We address this mismatch with Smooth Maximum Mean Discrepancy (SMMD), which builds on the classic MMD by incorporating value-distance kernels over numeric
tokens and graph-based smoothness. With this kernel defined over a numeric sub-vocabulary, SMMD aligns the predicted numeric distribution to the target via kernel matching and smooths the prediction-target residual over the induced kernel graph to encourage local consistency. We evaluate SMMD on four numeric-target tasks: mathematical
reasoning, arithmetic calculation, clock-time recognition, and chart question answering, across multiple open-weight LLM and VLM backbones. SMMD consistently improves accuracy over both cross-
entropy and recent numeric-target losses; analyses show complementary effects between MMD and smoothness and underscore the importance of distance-based kernel design. Code is available at https://github.com/Zuozhuo/smmd-loss.
Read the full abstract ▾
All authors Zhuo Zuo, Li Yue, Wenhao Zheng, Chenpeng Wang, Xianggen Liu
Computation and Language
Megan Wei, Deepali Aneja, Jiaqi Su et al. (+3 more)
Prosodic emphasis varies across
languages,
emotions, and speaking styles, yet existing emphasis detection models are largely trained and evaluated on monolingual neutral read
speech. We introduce MMEE (
Multilingual Multi-
Emotion Emphasis), a corpus of 10,000 professionally recorded expressive utterances (14.13 hours) across 7
languages and 34
emotion/style categories, with three-level perceptual labels (10 annotations per sample). We
benchmark two state-of-the-art architectures under monolingual, cross-lingual,
multilingual, cross-
emotion, cross-
dataset, and data-scale settings. Monolingual models show limited zero-shot transfer, degrading across typologically distant
languages, while
multilingual training substantially improves robustness. Models transfer robustly between high- and low-arousal
emotions; bidirectional transfer between synthetic and perceptual
benchmarks suggests shared prosodic structure; and performance stays robust even at smaller training scales.
Read the full abstract ▾
All authors Megan Wei, Deepali Aneja, Jiaqi Su, Yunyun Wang, Haonan Chen, Zeyu Jin
Computation and Language
Low-Agreeableness Persona Conditioning for Safe LLM Fine-Tuning
Austin MY Cheung and Yi Yang
Recent work has shown that fine-tuning large
language models (LLMs) for social warmth degrades factual reliability and increases sycophancy. We investigate a related but distinct failure mode: warmth fine-tuning also weakens
adversarial safety, making models more susceptible to jailbreaks and harmful output generation. We examine whether this reflects an inherent consequence of empathetic adaptation or an artifact of data construction. To address this, we introduce a persona-driven rewriting pipeline that conditions user turns on low agreeableness and pairs this with warm, de-escalating assistant responses. Across three experiments on four models, our approach reduces jailbreak susceptibility and harmful output rates relative to generic warmth fine-tuning baselines, while preserving conversational warmth. Representational probing provides suggestive evidence that this conditioning reduces the
geometric alignment between warmth and compliance directions in latent space. These results show that safer empathetic fine-tuning is achievable through data design alone, without safety labels, harm detectors, or changes to the training objective.
Read the full abstract ▾
Computation and Language
Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling
Changze Lv, Zhenghua Wang, Yiran Ding et al. (+9 more)
Large
Language Models (LLMs) still struggle with the “lost-in-the-middle” problem, where critical information located in the middle of long-context inputs is often underrepresented or lost. While existing methods attempt to address this by combining multi-scale rotary position
embeddings (RoPE), they typically suffer from high latency or rely on suboptimal hand-crafted scaling strategies. To overcome these limitations, we introduce a layer-specific positional
embedding scaling~(LPES) method that assigns distinct scaling factors to each layer. LPES achieves a more balanced
attention distribution without fine-tuning model parameters or increasing inference delay. A specially designed genetic
algorithm is employed to efficiently select the optimal scaling factors for each layer by incorporating Bézier curves to significantly reduce the search space. Extensive experiments demonstrate that LPES effectively mitigates positional
attention bias and delivers consistent improvements across multiple long-context
benchmarks, yielding up to an $11.2$% accuracy gain on the key-value
retrieval dataset.
Read the full abstract ▾
All authors Changze Lv, Zhenghua Wang, Yiran Ding, Yixin Wu, Tianlong Li, Zhibo Xu, Muling Wu, Tianyuan Shi, Shizheng Li, Qi Qian, Xuanjing Huang, Xiaoqing Zheng
Computation and Language · Cross-listed from cs.CV
Joint Transcription and Decryption of Images of Encrypted Handwritten Documents: A Comparison with the Traditional Pipeline
Marino Oliveros-Blanco, Lei Kang, Alicia Fornés et al. (+1 more)
Historical encrypted manuscripts present a challenging problem at the intersection of cryptology, linguistics, paleography, and computer
vision. Current automatic decipherment approaches usually rely on a two-stage pipeline: transcription of cipher symbols from manuscript images, followed by decryption into plaintext. However, this design is sensitive to transcription errors, which propagate to the final output. We present Direct Image Decryption, an end-to-end approach that directly maps encrypted manuscript images to plaintext, bypassing the intermediate transcription stage. Using the Copiale cipher as a case study, we build a synthetic data generation pipeline to create large-scale cipher-like training data and compare the traditional pipeline with the proposed joint architecture. Results show that joint image-to-plaintext modeling is a promising alternative to traditional transcription-based pipelines.
Read the full abstract ▾
All authors Marino Oliveros-Blanco, Lei Kang, Alicia Fornés, Beáta Megyesi
Computation and Language
Mitigating LLM-based p-Hacking by Preregistering for the Next LLM
Maria Thomas, Kristina Gligoric and Nihar B. Shah
Large
language models (LLMs) are increasingly used to generate, classify, and annotate data whose outputs feed downstream hypothesis tests. However, LLM-based research is easy to p-hack: a researcher can tune the
prompts, decoding parameters, or output format until a desired result is reached. We propose a protocol to mitigate p-hacking in LLM-based research: preregistering the experiment and eligible models, and then running it on the first eligible LLM that is released after the preregistration. The researcher finalizes the procedure on current models, preregisters the analysis plan together with a set of eligible future models, and runs the confirmatory analysis on the first eligible model released afterward. Because this model does not exist at commitment time, it cannot be hacked against; furthermore, configurations that hack one model frequently do not transfer to the next. We evaluate the protocol on two tasks whose true values are known. Across 20 models from four providers and 11 LLM-analysis configurations, the protocol would have blocked successful transfer of the p-hack in 73.9% and 72.7% of cases in the two tasks. Additional analyses reveal that mitigation remains substantial under several stress tests. Finally, putting money where our mouth is, we followed our own protocol and preregistered our experiment. The preregistered experiment confirmed the protocol's effectiveness: out of the 7 configurations that hacked the prior model, the hacking failed to carry over in 6 configurations on the first eligible model released afterward.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Textual Belief States for World Models: Identifiable Representation Learning Under Strict Mediation
Xiang Gao, Kaiwen Dong, Yuguang Yao et al. (+2 more)
World models in partially observed environments rely on latent representations that summarize interaction history, but in many modern LLM-based architectures predictive performance fails to reflect representation quality due to history bypass, rendering the latent state unidentifiable. Strict latent state mediation, requiring predictions to depend only on the latent state and action, is a classical principle that resolves this, but enforcing it in text-based settings is an open challenge: textual latent states are discrete and non-differentiable, precluding variational training, and expressive LLM decoders readily ignore the bottleneck. We show how to make strict mediation work in the text domain. We formalize why it is necessary, showing that strict mediation makes representation quality empirically testable while history-leaky architectures break this connection. We then introduce textual latent states, which are discrete, interpretable, and variable-length, and factorized GRPO (fGRPO), a tree-structured reinforcement learning method that enforces strict mediation during training. Experiments on TextWorld and ScienceWorld show preserved one-step prediction accuracy alongside up to 57% gains in representation quality and 98% improvements in rollout performance, increasing with task complexity and horizon.
Read the full abstract ▾
All authors Xiang Gao, Kaiwen Dong, Yuguang Yao, Padmaja Jonnalagedda, Kamalika Das
Computation and Language
From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language Models
Ponhvoan Srey, Xiaobao Wu, Cong-Duy Nguyen et al. (+3 more)
Probe-based uncertainty estimation (UE) has emerged as a prominent approach to detect
hallucinations in Large
Language Models (LLMs) by learning uncertainty from internal model signals. Yet, recent methods vary simultaneously across feature design, training data construction, and evaluation setting, obscuring what actually drives performance. To address this issue, we propose a factorised study of probe-based UE under matched conditions. Our results show that raw hidden states and
attention features are difficult to outperform in-domain. However, under distribution shift, structured and compressed features are more robust, suggesting that in-domain performance alone is insufficient to measure progress. Furthermore, prompting and label construction significantly affect probe behaviour. Building on these empirical design principles, we construct
benchmark-pretrained probes that retain useful discriminative signal on open-ended factual generation. Their competitive performance relative to limited-label target setting probes and sampling-based
estimators suggests that they can provide a practical starting point for subsequent target-domain adaptation. More broadly, our work encourages deployment-oriented evaluation of probe-based uncertainty
estimators.
Read the full abstract ▾
All authors Ponhvoan Srey, Xiaobao Wu, Cong-Duy Nguyen, Quang Minh Nguyen, Duc Anh Vu, Anh Tuan Luu
Computation and Language
When Search Agents Should Ask: DiscoBench for Clarification-Aware Deep Search
Yiling Tao, Shihan Deng, Meiling Tao et al. (+3 more)
Search
agents powered by large
language models (LLMs) are increasingly used to solve complex information-seeking tasks, requiring multi-step
retrieval and
reasoning to fulfill user goals. However, existing
benchmarks often assume that user queries are complete and explicit, overlooking the fact that real-world search requests are frequently vague, underspecified, or even factually incorrect. In deep search scenarios, such ambiguity can propagate along multi-step
reasoning chains and lead
agents toward incorrect search trajectories. To address this gap, we introduce DiscoBench, a
benchmark for clarification-aware deep search, designed to evaluate whether search
agents can proactively identify ambiguity, ask effective clarification questions, and recover correct
reasoning paths through user interaction. DiscoBench contains 211 samples and 463 ambiguity instances across 11 real-world domains, covering four ambiguity types. We further design a user simulator for multi-turn interaction and evaluate model performance from four perspectives: task utility, ambiguity detection, interaction strategy, and cost efficiency. Experiments on representative LLMs show that ambiguity detection and effective clarification are distinct capabilities, and that repeatedly searching instead of asking for clarification often performs worse than direct guessing, highlighting a critical gap between
retrieval ability and interactive problem-solving in current search
agents.
Read the full abstract ▾
All authors Yiling Tao, Shihan Deng, Meiling Tao, Pengzhi Wei, Zhichao Hu, Zhihao Zhu
Computation and Language
Yuvion LLM: An Adversarially-Aware Large Language Model for Content And AI Safety
Ting Ma, Xiufeng Huang, Benlei Cui et al. (+43 more)
As large
language models are increasingly deployed in real-world systems, safety failures can still lead to harmful outputs and dangerous misuse. We argue that the essence of safety is
adversarial: many failures arise not from natural inputs alone, but from strategic attempts to evade model policies and safeguards. However, existing general-purpose model development largely overlook this
adversarial nature, and often remain insufficient for realistic safety scenarios involving planning, tool use, and multi-step
reasoning, causing measured safety performance to overestimate real deployment robustness. To address this gap, we present Yuvion LLM, a large
language model built for adversarially robust content safety and broader AI safety. Yuvion LLM treats
adversarial robustness and agentic capability as first-class objectives. Its pipeline combines adversarially aware data construction, knowledge-enhanced continued pretraining, and policy-grounded multi-task safety post-training, including risk-aware supervised fine-tuning and reinforcement learning-based policy optimization, together with safety-aware agentic reinforcement learning for tool use and multi-step
reasoning in complex safety scenarios. We further introduce the Yuvion LLM RiskEval (YLRE), a collection of 93
benchmarks across four evaluation categories, covering diverse open and internal evaluations with a focus on safety,
adversarial robustness, and real-world capability requirements. Across these evaluations, Yuvion LLM demonstrates clear advantages on safety-focused
benchmarks and particularly strong robustness under
adversarial conditions, while maintaining solid overall capability. Notably, Yuvion-8B outperforms most state-of-the-art baselines, including substantially larger models such as GPT-5.4 and Qwen3-MAX, on several safety tasks.
Read the full abstract ▾
All authors Ting Ma, Xiufeng Huang, Benlei Cui, Xiaowen Xu, Shikai Qiu, Ruijie Jian, Hongxing Li, Guanghui Wang, Longtao Huang, Haiwen Hong, Haolei Xu, Wenjing Jiang, Ziwen Xu, Zhaoyu Fan, Shaoxuan He, Chuxi Xiao, Yujian Li, Xinyue Chen, Chunyang Chai, Wenxuan Liu, Ziheng Wang, Dongjie Zhang, Yangfan Zhou, Libin Dong, Yupeng Cao, Xiaoqian Xia, Jing Wang, Zhe Jiang, Zhenan Ye, Guang Yang, Bin Liu, Wei Peng, Ziqiang Zhu, Meihui Lian, Kaiwen Lv Kacuila, Haidong Ding, Bingyu Zhu, Yan Wang, Hai Zhao, Xuan Jin, Wei Zhao, Pengfei Sun, Wei Wang, Huiming Zhang, Bin Li, Hui Xue
Computation and Language
Cross-Platform Chinese Offensive Comment Detection via Dual-Threshold Hard Example Mining
Ruixing Ren, Junhui Zhao and Fangfang Wang
Cross-platform deployment of offensive comment detection for Chinese social media suffers performance degradation. The paper proposes a dual-threshold hard mining method to address this. First, the clean-Chinese-base RoBERTa is finetuned on COLD to establish a binary baseline for fair comparison. Second, a three-class fine-labeled test set covering Weibo, Xiaohongshu, Tieba, and Zhihu is constructed, domain distances from the source are quantified using Jaccard and Proxy-A Distance, as well as the degradation bottleneck of the baseline under domain shift is systematically revealed. Herein, a dual threshold hard example mining strategy is proposed. High- and low-confidence error-prone samples are filtered from unlabeled corpora by prediction confidence. The model is secondarily finetuned under implicit contexts with merely a small set of manually labeled hard examples, realizing low-cost cross-platform domain adaptation. Experiments reveal significant performance gains of the optimized model across four platforms.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners Insights from Online Forums
Dana Rezazadegan, Atie Kia, Phongpadid Nandavong et al. (+6 more)
Dyslexic learners increasingly use artificial intelligence (AI) tools to support reading, writing, organisation, and study-related tasks. However, their lived experiences with these tools remain largely underexamined. This paper proposes DysLexLens, a low-resource LLM framework, designed to analyse dyslexic learners experience with AI through online forum discussions. DysLexLens is designed as an end-to-end, evidence-traceable architecture which transforms noisy social media posts into a dictionary-driven corpora, provides knowledge-graph (KG)-based question
reasoning, generates verifiable query responses, and enables response evaluation through quantitative and human-grounded assessment. DysLexLens has four key features. First, it employs a dictionary-driven filtering method to construct a more focused Reddit corpus on dyslexia and AI, filtering out noisy and weakly related posts to improve the relevance of data collected from low-resource forum contexts. Second, it integrates LLM-assisted semantic analysis with KG-based query
reasoning to uncover meaningful patterns. Third, it has quantitative evaluation metrics (RAGAS and Query Robustness) to measure LLM-generated response performance. Fourth, it provides structured qualitative validation guidelines for assessing response quality, with a specific focus on
hallucination and evidence
alignment. We demonstrate the effectiveness of DysLexLens using dyslexia-related Reddit forum data and 30 questions. The results show its potential generalisability to other low-resource forum data contexts. DysLexLens, sample data, questions and evaluation results are available at Github to support reproducibility.
Read the full abstract ▾
All authors Dana Rezazadegan, Atie Kia, Phongpadid Nandavong, Dominique Carlon, Jeremy Nguyen, Abhik Banerjee, James Marshall, Anthony McCosker, Yong-Bin Kang
Computation and Language
Masked Language Flow Models
Iskander Azangulov, Kianoosh Ashouritaklimi, Leo Zhang et al. (+2 more)
Masked
Diffusion Models (MDMs) promise fast, parallel
language generation, but their reverse transition factorises across
token positions – an approximation that breaks down in the few-step sampling regime where parallel generation ought to provide the greatest efficiency gains. Flow
Language Models (FLMs) sidestep this limitation by learning a continuous flow that transports noise toward clean sequences represented in Euclidean space, inducing a flow map that can be distilled for single-step generation. However, this makes complex tasks requiring multi-step
reasoning problematic for FLMs, as FLMs are forced to decode every
token during generation. To address this, we introduce Masked
Language Flow Models (MLFMs), which incorporate masking into FLMs using a continuous
stochastic interpolant to bridge partially masked and clean sequences. This design enables conditional generation via continuous flows and allows pretrained MDMs to be converted into MLFMs through a simple, lightweight adaptation. Leveraging this flexibility, we propose a novel sampler that alternates continuous denoising with the discrete unmasking of confident
tokens to better support multi-step
reasoning. We evaluate our approach on GSM8K and MT-Bench and find, for the first time, that flow-based
language models can be scaled to solve downstream
reasoning and instruction-following tasks.
Read the full abstract ▾
All authors Iskander Azangulov, Kianoosh Ashouritaklimi, Leo Zhang, Simon Vary, Patrick Rebeschini