Computation and Language
Towards Understanding Continual Factual Knowledge Acquisition of Language Models: From Theory to Algorithm
Haoyu Wang, Yifan Shang, Zhongxiang Sun et al. (+3 more)
Continual Pre-Training (CPT) is essential for enabling
Language Models (LMs) to integrate new knowledge without erasing old. While classical CPT techniques like data replay have become the standard paradigm, the mechanisms underlying how LMs acquire and retain facts over time, termed as continual Factual Knowledge Acquisition (cFKA), remain unclear. In this work, we present a theoretical framework that characterizes the training dynamics of cFKA using a single-layer
Transformer, offering a unified explanation for the behavior of representative CPT methods. Our analysis reveals that regularization-based methods merely adjust the convergence rate of parameters without altering the inherent forgetting tendency, whereas data replay methods succeed in shifting convergence dynamics and stabilizing pretrained knowledge. Building on these insights, we propose a novel generative data replay approach, called Selecting Tokens via attentiOn Contribution~(STOC), which identifies influential factual snippets to guide replay data generation. Extensive experiments on both synthetic and real-world
datasets validate our findings and demonstrate that STOC effectively enhances cFKA by mitigating catastrophic forgetting.
Read the full abstract ▾
All authors Haoyu Wang, Yifan Shang, Zhongxiang Sun, Weijie Yu, Xiao Zhang, Jun Xu
Computation and Language
Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs
Krishak Aneja, Manas Mittal, Anmol Goel et al. (+2 more)
Fine-tuning Large
Language Models (LLMs) on benign narrow data can sometimes induce broad harmful behaviors, a vulnerability termed emergent misalignment (EM). While prior work links these failures to specific directions in the activation space, their relationship to the model's broader persona remains unexplored. We map the latent personality space of LLMs through established psychometric profiles like the Big Five, Dark Triad, and LLM-specific behaviors (e.g. evil, sycophancy), and show that the semantic geometry is highly stable across aligned models and their corrupted fine-tunes. Through causal interventions, we find that directions isolating social valence, such as the 'Evil' persona vector, and a Semantic Valence Vector (SVV) that we introduce, function as intrinsic guardrails: ablating them drives the misalignment rates above $40$%, while amplifying them suppresses the failure mode to less than $3$%. Leveraging the structural stability of the personality space, we show that vectors extracted $\textit{a priori}$ from an instruct-tuned model transfer zero-shot to successfully regulate EM in corrupted fine-tunes. Overall, our findings suggest that harmful fine-tuning does not overwrite a model's internal representation of personality, allowing conserved representations to serve as robust, cross-distribution guardrails.
Read the full abstract ▾
All authors Krishak Aneja, Manas Mittal, Anmol Goel, Ponnurangam Kumaraguru, Vamshi Krishna Bonagiri
Computation and Language
Interpretable Coreference Resolution Evaluation Using Explicit Semantics
Bruno Gatti, Giuliano Martinelli and Roberto Navigli
Coreference resolution is typically evaluated using aggregate statistical metrics such as CoNLL-F1, which measure structural overlap between predicted and gold clusters. While widely used, these metrics offer limited diagnostic insights, penalizing errors without revealing whether a system struggles with specific semantic categories, such as people, locations, or events, and making it difficult to interpret model capabilities or derive actionable improvements. We address this gap by introducing a semantically-enhanced evaluation framework for coreference resolution. Our approach overlays Concept and Named Entity Recognition (CNER) onto coreference outputs, assigning semantic labels to nominal mentions and propagating them to entire coreference clusters. This enables the computation of typed scores aimed at evaluating mention extraction and linking capabilities stratified by semantic class. Across our experiments on OntoNotes, LitBank, and PreCo, we show that our framework uncovers systematic weaknesses that remain obscured by aggregate metrics. Furthermore, we demonstrate that these diagnostics can be used to design targeted, low-cost data augmentation strategies, achieving measurable out-of-domain improvements.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image
Alan Arazi, Eilam Shapira, Shoham Grunblat et al. (+8 more)
Tabular Foundation Models have recently established the state of the art in supervised tabular learning, by leveraging pretraining to learn generalizable representations of numerical and categorical structured data. However, they lack native support for unstructured modalities such as text and image, and rely on frozen, pretrained
embeddings to process them. On established
Multimodal Tabular Learning
benchmarks, we show that tuning the
embeddings to the task improves performance. Existing
benchmarks, however, often focus on the mere co-occurrence of modalities; this leads to high variance across
datasets and masks the benefits of task-specific tuning. To address this gap, we introduce MulTaBench, a
benchmark of 40
datasets, split equally between image-tabular and text-tabular tasks. We focus on predictive tasks where the modalities provide complementary predictive signal, and where generic
embeddings lose critical information, necessitating Target-Aware Representations that are aligned with the task. Our experimental results demonstrate that the gains from target-aware representation tuning generalize across both text and image modalities, several tabular learners, encoder scales, and
embedding dimensions. MulTaBench constitutes the largest image-tabular benchmarking effort to date, spanning high-impact domains such as healthcare and e-commerce. It is designed to enable the research of novel architectures which incorporate joint modeling and target-aware representations, paving the way for the development of novel
Multimodal Tabular Foundation Models.
Read the full abstract ▾
All authors Alan Arazi, Eilam Shapira, Shoham Grunblat, Mor Ventura, Elad Hoffer, Gioia Blayer, David Holzmüller, Lennart Purucker, Gaël Varoquaux, Frank Hutter, Roi Reichart
Computation and Language
Responsible Benchmarking of Fairness for Automatic Speech Recognition
Felix Herron, Ange Richard, François Portet et al. (+2 more)
Many studies have shown automatic
speech processing (ASR) systems have unequal performance across speakergroups (SG's). However, the manner in which such studies arrive at this conclusion is inconsistent. To pave the wayfor more reliable results in future studies, we lay out best practices for benchmarking ASR
fairness based on literaturefrom machine learning
fairness, social sciences, and
speech science. We first describe the importance of preciselythe
fairness hypothesis being interrogated, and tailoring
fairness metrics to apply specifically to said hypothesis.We then examine several
benchmarks used to rate ASR systems on
fairness and discuss how their results can bemisconstrued without assiduous oversight into the intersections between SG's. We find that evaluating fairnessbased on single heterogeneous SG's, such as they are defined in
fairness benchmarks, can lead to misidentifyingwhich SG's are actually being mistreated by ASR systems. We advocate for as fine-grained an analysis as possibleof the intersectionality of as many demographic variables as are available in the metadata of
fairness corpora in orderto tease out such spurious correlations
Read the full abstract ▾
All authors Felix Herron, Ange Richard, François Portet, Alexandre Allauzen, Solange Rossato
Computation and Language
Measuring Embedding Sensitivity to Authorial Style in French: Comparing Literary Texts with Language Model Rewritings
Benjamin Icard, Lila Sainero, Alice Breton et al. (+2 more)
Large
language models (LLMs) can convincingly imitate human writing styles, yet it remains unclear how much stylistic information is encoded in
embeddings from any
language model and retained after LLM rewriting. We investigate these questions in French, using a controlled literary
dataset to quantify the effect of stylistic variation via changes in
embedding dispersion. We observe that
embeddings reliably capture authorial stylistic features and that these signals persist after rewriting, while also exhibiting LLM-specific patterns. These analytical results offer promising directions for authorship imitation detection in the era of
language models.
Read the full abstract ▾
All authors Benjamin Icard, Lila Sainero, Alice Breton, Evangelia Zve, Jean-Gabriel Ganascia
Computation and Language
Where do aspectual variants of light verb constructions belong?
Aggeliki Fotopoulou, Eric Laporte and Takuya Nakamura
Expressions with an aspectual variant of a light verb, e.g. 'take on debt' vs. 'have debt', are frequent in texts but often difficult to classify between verbal idioms, light verb constructions or compositional phrases. We investigate the properties of such expressions with a disputed membership and propose a selection of features that determine more satisfactory boundaries between the three categories in this zone, assigning the expressions to one of them.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
LLARS: Enabling Domain Expert & Developer Collaboration for LLM Prompting, Generation and Evaluation
Philipp Steigerwald, Mara Stieler, Jennifer Burghardt et al. (+2 more)
We demonstrate LLARS (LLM Assisted Research System), an open-source platform that bridges the gap between domain experts and developers for building LLM-based systems. It integrates three tightly connected modules into an end-to-end pipeline: Collaborative
Prompt Engineering for real-time co-authoring with version control and instant LLM testing, Batch Generation for configurable output production across user-selected
prompts $\times$ models $\times$ data with cost control, and Hybrid Evaluation where human and LLM evaluators jointly assess outputs through diverse assessment methods, with live agreement metrics and provenance analysis to identify the best model-
prompt combination for a given use case. New
prompts and models are automatically available for batch generation and completed batches can be turned into evaluation scenarios with a single click. Interviews with six domain experts and three developers in online counselling confirmed that LLARS feels intuitive, saves considerable time by keeping everything in one place and makes interdisciplinary collaboration seamless.
Read the full abstract ▾
All authors Philipp Steigerwald, Mara Stieler, Jennifer Burghardt, Eric Rudolph, Jens Albrecht
Computation and Language
VISTA: A Controllable Platform for Generating and Auditing Egocentric Assistance Scenarios
Yu-Hsiang Liu, Yu-Chien Tang and An-Zi Yen
Evaluating whether AI
agents can proactively assist humans in daily activities, ranging from routine household tasks to urgent safety-critical situations, requires diverse visual data. However, collecting such scenarios in the real world is often difficult, costly, or unsafe, and simulation environments often lack the social commonsense needed to simulate the consequences of different actions. In this work, we present VISTA, a controllable platform that uses a user-provided scenario seed, defined as a short natural-
language description of the intended assistance situation, to generate editable plans,
egocentric videos, and an auditable review trail. VISTA structures scenario intent around three interaction modes, including reactive, explicit proactive, and implicit proactive, and two consequence families, including safety-critical and everyday inconvenience, with no-assistance cases as controls. Its six-stage pipeline exposes the design brief, timed event script, first-frame plan, motion plan, and
video plan, allowing users to revise each artifact in natural
language before explicitly authorizing media generation. A human evaluation shows that
videos retained by the complete VISTA workflow align more closely with their scenario seeds than outputs from two one-pass baselines. VISTA thereby makes targeted
egocentric scenario generation inspectable, revisable, and empirically auditable.
Read the full abstract ▾
Computation and Language
ThreatCore: A Benchmark for Explicit and Implicit Threat Detection
Davide Bruni, Carlo Bardazzi and Maurizio Tesconi
Threat detection in Natural
Language Processing lacks consistent definitions and standardized
benchmarks, and is often conflated with broader phenomena such as toxicity, hate
speech, or offensive
language. In this work, we introduce ThreatCore, a public available
benchmark dataset for fine-grained threat detection that distinguishes between explicit threats, implicit threats, and non-threats. The
dataset is constructed by aggregating multiple publicly available resources and systematically re-annotating them under a unified operational definition of threat, revealing substantial inconsistencies across existing labels. To improve the coverage of underrepresented cases, particularly implicit threats, we further augment the
dataset with synthetic examples, which are manually validated using the same annotation protocol adopted for the re-annotation of the public
datasets, ensuring consistency across all data sources. We evaluate Perspective API, zero-shot classifiers, and recent
language models on ThreatCore, showing that implicit threats remain substantially harder to detect than explicit ones. Our results also indicate that incorporating Semantic Role Labeling as an intermediate representation can improve performance by making the structure of harmful intent more explicit. Overall, ThreatCore provides a more consistent
benchmark for studying fine-grained threat detection and highlights the challenges that current models still face in identifying indirect expressions of harmful intent.
Read the full abstract ▾
Computation and Language
ICT-NLP at SemEval-2026 Task 3: Less Is More – Multilingual Encoder with Joint Training and Adaptive Ensemble for Dimensional Aspect Sentiment Regression
Liyuan Huang, Jiawei He, Wutao Shen et al. (+2 more)
This paper describes our system to SemEval-2026 Task 3 Track A Subtask 1 on Dimensional Aspect Sentiment Regression (DimASR). We propose a lightweight and resource-efficient system built entirely on
multilingual pre-trained encoders, without relying on LLMs or external corpora. We adopt joint
multilingual and multi-domain training to facilitate cross-lingual transfer and alleviate data sparsity, introduce a bounded regression transformation that improves training stability while constraining predictions within the valid range, and employ an adaptive ensemble strategy via subset search to reduce prediction variance. Experimental results demonstrate that our system achieves strong and consistent performance, ranking 1st on zho-res, 2nd on zho-lap, and 3rd on jpn-hot, with all remaining
datasets placed within the top half of participating teams.
Read the full abstract ▾
All authors Liyuan Huang, Jiawei He, Wutao Shen, Lin Li, Jin Zhang
Computation and Language
Multi-domain Multi-modal Document Classification Benchmark with a Multi-level Taxonomy
Denghao Ma, Qing Liu, Zulong Chen et al. (+5 more)
Document classification forms the backbone of modern enterprise content management, yet existing
benchmarks remain trapped in oversimplified paradigms – single domain settings with flat label structures – that bear little resemblance to the hierarchical, multi-modal, and cross-domain nature of real-world business documents. This gap not only misrepresents practical complexity but also stifles progress toward industrially viable document intelligence. To bridge this gap, we construct the first Multi-level, Multi-domain, Multi-modal document classification
Benchmark (MMM-Bench). MMM-Bench includes (1) a deeply hierarchical taxonomy spanning five levels that capture the authentic organizational logic of business documentation; and (2) 5,990 real-world multi-modal documents meticulously curated from 12 commercial domains in Alibaba. Each document is manually annotated with a complete hierarchical path by domain experts. We establish comprehensive baselines on MMM-Bench, which consists of open-weight models and API-based models. Through systematic experiments, we identify four fundamental challenges within MMM-Bench and propose corresponding insights. To provide a solid foundation for advancing research in multi-level, multi-domain document classification, we release all of the data and the evaluation toolkit of MMM-Bench at https://github.com/MMMDC-Bench/MMMDC-Bench.
Read the full abstract ▾
All authors Denghao Ma, Qing Liu, Zulong Chen, Chuanfei Xu, Jia Xu, Zhibo Yang, Wei Shao, Zhao Li
Computation and Language
Where Does Long-Context Supervision Actually Go? Effective-Context Exposure Balancing
Jinchang Zhu, Jindong Li, Chengyu Zou et al. (+4 more)
Long-context adaptation is often viewed as window scaling, but this misses a
token-level supervision mismatch: in packed training with document masking, each target
token's effective context remains short. We introduce EXACT, a supervision-allocation objective that assigns extra weight to long effective-context targets by inverse frequency within the long tail. Across seven Qwen/LLaMA CPT configurations, EXACT improves all 28 trained/extrapolated NoLiMa and RULER comparisons. On Qwen2.5-0.5B, NoLiMa improves by +10.09 (trained) and +5.34 (extrapolated); RULER by +10.69 and +5.55. On LLaMA-3.2-3B, RULER improves by +17.91 and +16.11. Standard QA/
reasoning are preserved (+0.24 macro change across six
benchmarks). A distance-resolved probe shows gains arise when evidence is thousands of
tokens away, while short cases remain unchanged. Results support a supervision-centric thesis: long-context adaptation depends on how strongly training supervises long-context predictions.
Read the full abstract ▾
All authors Jinchang Zhu, Jindong Li, Chengyu Zou, Rong Fu, Chao Wang, Haowei He, Menglin Yang
Computation and Language
Mela: Test-Time Memory Consolidation based on Transformation Hypothesis
Lungchuan Chen
Memory consolidation, the process by which transient experiences are transformed into stable, structured representations, is a foundational organizing principle in the human
brain, yet it remains largely unexplored as a design principle for modern sequence models. In this work, we leverage established neuroscientific theories of
memory consolidation and cross-frequency coupling to propose the Hierarchical
Memory Module (HMM), a neural
memory architecture composed of two functionally distinct sub-modules that operate at different update frequencies. Inspired by the transformation hypothesis, the low-frequency sub-module produces high-level representations that capture abstract, gist-level knowledge, while the high-frequency sub-module produces fine-grained representations that preserve richer episodic detail. The final
memory output is dynamically reconstructed as a context-dependent combination of both representations, analogous to the reconstructive nature of human
memory retrieval. We integrate HMM into a
Transformer-based
language decoder to form Mela, a family of
memory-augmented
language models that perform online
memory consolidation at test time. To further exploit the multi-granularity
memory representations produced by HMM, we introduce MemStack, a method that distributes different levels of
memory features across the early layers of the decoder without introducing additional
tokens. Experiments on
language modeling demonstrate that Mela outperforms
Transformer baselines across all the model sizes. Moreover, with the pretrained context length fixed at 4K, Mela maintains performance on significantly longer contexts, whereas
Transformer baselines degrade rapidly beyond their training length. Extensive ablation studies validate the contribution of each component and provide guidance for practical configuration.
Read the full abstract ▾
Computation and Language · Cross-listed from cond-mat.stat-mech
Collective Alignment in LLM Multi-Agent Systems: Disentangling Bias from Cooperation via Statistical Physics
Cristiano De Nobili
We investigate the emergent collective dynamics of LLM-based multi-
agent systems on a 2D square
lattice and present a model-agnostic statistical-physics method to disentangle social conformity from intrinsic bias, compute critical exponents, and probe the collective behavior and possible phase transitions of multi-
agent systems. In our framework, each node of an $L\!\times\!L$
lattice hosts an identical LLM
agent holding a binary state ($+1$/$-1$, mapped to yes/no) and updating it by querying the model conditioned on the four nearest-neighbor states. The sampler temperature $T$ serves as the sole control parameter. Across three open-weight models (llama3.1:8b, phi4-mini:3.8b, mistral:7b), we measure magnetization and susceptibility under a global-flip protocol designed to probe $\mathbb{Z}_2$
symmetry. All models display temperature-driven order-disorder crossovers and susceptibility peaks; finite-size scaling on even-$L$
lattices yields effective exponents $\gamma/\nu$ whose values are model-dependent, close to but incompatible with the 2D Ising universality class ($\gamma/\nu=7/4$). Our method enables the extraction of effective $\beta$-weighted couplings $\tilde{J}(T)$ and fields $\tilde{h}(T)$, which serve as a measure of social conformity and intrinsic bias. In the models we analyzed, we found that collective
alignment is dominated by an intrinsic bias ($\tilde{h}\gg\tilde{J}$) rather than by cooperative neighbor coupling, producing field-driven crossovers instead of genuine phase transitions. These effective parameters vary qualitatively across models, providing compact collective-behavior fingerprints for LLM
agents and a quantitative diagnostic for the reliability of multi-
agent consensus and collective
alignment.
Read the full abstract ▾
Computation and Language
Jaehoon Yoo, Wonjung Kim, Chanhyuk Lee et al. (+1 more)
Masked
Diffusion Models (MDMs) have emerged as a promising alternative to autoregressive models in
language modeling, offering the advantages of parallel decoding and bidirectional context processing within a simple yet effective framework. Specifically, their explicit distinction between masked
tokens and data underlies their simple framework and effective conditional generation. However, MDMs typically require many sampling iterations due to factorization errors stemming from simultaneous
token updates. We observe that a theoretical lower bound of the factorization error exists, which standard MDMs cannot reduce due to their use of a deterministic single-state mask. In this paper, we propose the Infinite Mask
Diffusion Model (IMDM), which introduces a
stochastic infinite-state mask to mitigate the theoretical bound while directly inheriting the benefits of MDMs, including the compatibility with pre-trained weights. We empirically demonstrate that MDM fails to perform few-step generation even in a simple synthetic task due to the factorization error bound, whereas IMDM can find an efficient solution for the same task. Finally, when equipped with appropriate
distillation methods, IMDM surpasses existing few-step
distillation methods at small step counts on LM1B and OpenWebText. Code is available at https://Ugness.github.io/official_imdm.
Read the full abstract ▾
All authors Jaehoon Yoo, Wonjung Kim, Chanhyuk Lee, Seunghoon Hong
Computation and Language
Learning Less Is More: Premature Upper-Layer Attention Specialization Hurts Language Model Pretraining
Jinchang Zhu, Jindong Li, Yuwen Hao et al. (+3 more)
A causal-decoder block is hierarchical: lower layers build the residual basis that upper layers attend over. We identify a failure mode in GPT pretraining: upper layers commit to sharp
attention patterns before lower-layer features stabilize. We call this premature upper-layer
attention specialization. Temporarily slowing only upper-layer Q/K projections during early training improves final perplexity and downstream accuracy without altering other parameters; it prevents upper
attention from collapsing onto an immature residual basis. In LLaMA-style blocks, the same intervention is nearly unnecessary. Through ablations, we isolate multiplicative gated FFNs (not RMSNorm or bias removal) as the component that suppresses the upstream residual writes driving the failure. A pathwise analysis unifies both findings: the learning-rate intervention reduces a step-size factor, while gated FFNs reduce a residual-energy factor on the same growth pathway. Our results identify upper-layer Q/K timing as a concrete interaction point between decoder architecture and optimization.
Read the full abstract ▾
All authors Jinchang Zhu, Jindong Li, Yuwen Hao, Chengyu Zou, Rong Fu, Menglin Yang
Computation and Language
DeepRefine: Agentic Knowledge Refinement via Reinforcement Learning
Haoyu Huang, Jiaxin Bai, Shujie Liu et al. (+7 more)
External knowledge enables large
language model (LLM)
agents to ground their actions and decisions beyond intrinsic parametric
memory in open-ended, knowledge-intensive downstream tasks. Yet the quality of the underlying knowledge bases is systematically limited by incompleteness, incorrectness, or redundancy, manifested as missing evidence or cross-document links, low-confidence or imprecise claims, and ambiguous or coreference resolution issues. Such defects compound under iterative use, degrading
retrieval fidelity and downstream task performance. We present DeepRefine, a reinforcement learning framework for agentic knowledge refinement that evolves the quality of any pre-constructed structured knowledge bases, e.g., knowledge graphs or LLM-Wikis, with user queries to make it more suitable for the downstream tasks. DeepRefine performs multi-turn interactions with the knowledge base and conducts abductive diagnosis over the interaction history, localizes likely defects, and executes targeted refinement actions for incremental knowledge base updates. To further optimize refinement policies of DeepRefine without golden refinement trajectories, we introduce a Gain-Beyond-Draft (GBD) reward and train the
reasoning process end-to-end via reinforcement learning. Extensive experiments demonstrate consistent downstream gains over strong baselines.
Read the full abstract ▾
All authors Haoyu Huang, Jiaxin Bai, Shujie Liu, Yang Wei, Huihao Jing, Hong Ting Tsang, Yisen Gao, Zhongwei Xie, Yufei Li, Yangqiu Song
Computation and Language
Coherency through formalisations of Structured Natural Language, A case study on FRETish
Joost J. Joosten, Marina López Chamosa and Sofía Santiago Fernández
Formalisation is the process of writing system requirements in a formal
language. These requirements mostly originate in Natural
Language. In the field of Formal Methods, formalisation is often identified as one of the most delicate and complicated steps in the verification process. Not seldomly, formalisation tools and environments choose various levels of requirement descriptions: Natural
Language, Technical
Language, Diagram Representations and Formal
Language, to mention a few. In the literature, there are various maxims and principles of good practice to guide the process of requirement formalisation. In this paper we propose a new guideline: Coherency through Formalisations. The guideline states that the different levels of formalisation mentioned above should roughly follow the same logical structure. The principle seems particularly relevant in the setting where LLMs are prompted to perform
reasoning tasks that can be checked by formal tools using Structured Natural
Language to act as an intermediate layer bridging both paradigms. In the light of coherency, we analyze NASA's Formal Requirement Elicitation Tool FRET and propose an alternative automated translation of the Controlled Natural
Language FRETish to the formal
language of MTL. We compare our translation to the original translation and prove equivalence using model checking. Some statistics are performed which seem to favor the new translation. As expected, the translation process yielded interesting reflections and revealed inconsistencies which we present and discuss.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
SlimSpec: Low-Rank Draft LM-Head for Accelerated Speculative Decoding
Anton Plaksin, Sergei Krutikov, Sergei Skvortsov et al. (+1 more)
Speculative decoding speeds up autoregressive generation in Large
Language Models (LLMs) through a two-step procedure, where a lightweight draft model proposes
tokens which the target model then verifies in a single forward pass. Although the drafter network is small in modern architectures, its LM-head still performs projection to a large vocabulary, becoming one of the major computational bottlenecks. In prior work this issue has been predominantly addressed via static or dynamic vocabulary truncation. Yet mitigating the bottleneck, these methods bring in extra complexity, such as special vocabulary curation, sophisticated inference-time logic or modifications of the training setup. In this paper, we propose SlimSpec, a low-rank parameterization of the drafter's LM-head that compresses the inner representation rather than the output, preserving full vocabulary support. We evaluate our method with EAGLE-3 drafter across three target models and diverse
benchmarks in both latency- and throughput-bound inference regimes. SlimSpec achieves $4\text{-}5\times$ acceleration over the standard LM-head architecture while maintaining a competitive acceptance length, surpassing existing methods by up to $8\text{-}9\%$ of the end-to-end speedup. Our method requires minimal adjustments of training and inference pipelines. Combined with the aforementioned speedup improvements, it makes SlimSpec a strong alternative across wide variety of draft LM-head architectures.
Read the full abstract ▾
All authors Anton Plaksin, Sergei Krutikov, Sergei Skvortsov, Alexander Samarin
Computation and Language · Cross-listed from cs.CY
StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs
Pierre Le Jeune, Étienne Duchesne, Weixuan Xiao et al. (+4 more)
Multilingual studies of social bias in open-ended LLM generation remain limited: most existing
benchmarks are English-centric, template-based, or restricted to recognizing pre-specified stereotypes. We introduce StereoTales, a
multilingual dataset and evaluation pipeline for systematically studying the emergence of social bias in open-ended LLM generation. The
dataset covers 10
languages and 79 socio-demographic attributes, and comprises over 650k stories generated by 23 recent LLMs, each annotated with the socio-demographic profile of the protagonist across 19 dimensions. From these, we apply statistical tests to identify more than 1{,}500 over-represented associations, which we then rate for harmfulness through both a panel of humans (N = 247) and the same LLMs. We report three main findings. (i) Every model we evaluate emits consequential harmful stereotypes in open-ended generation, regardless of size or capabilities, and these associations are largely shared across providers rather than isolated misbehaviors. (ii)
Prompt language strongly shapes which stereotypes appear: rather than transferring as a shared set of biases, harmful associations adapt culturally to the
prompt language and amplify bias against locally salient protected groups. (iii) Human and LLM harmfulness judgments are broadly aligned (Spearman $\rho=0.62$), with disagreements concentrating on specific attribute classes rather than specific providers. To support further analyses, we release the evaluation code and the
dataset, including model generations, attribute annotations, and harmfulness ratings.
Read the full abstract ▾
All authors Pierre Le Jeune, Étienne Duchesne, Weixuan Xiao, Stefano Palminteri, Bazire Houssin, Benoît Malézieux, Matteo Dora
Computation and Language
Can Language Models Analyze Data? Evaluating Large Language Models for Question Answering over Datasets
Andreas Xenofontos and Pavlos Fafalios
This paper investigates the effectiveness of large
language models (LLMs) in answering questions over
datasets. We examine their performance in two scenarios: (a) directly answering questions given a
dataset file as input, and (b) generating SQL queries to answer questions given the schema of a relational database. We also evaluate the impact of different prompting strategies on model performance. The study includes both state-of-the-art LLMs and smaller
language models that require fewer resources and operate at lower computational and financial cost. Experiments are conducted on two
datasets containing questions of varying difficulty. The results demonstrate the strong performance of large LLMs, while highlighting the limitations of smaller, more cost-efficient models. These findings contribute to a better understanding of how LLMs can be utilized in data analytics tasks and their associated limitations.
Read the full abstract ▾
Computation and Language
Aligning LLM Uncertainty with Human Disagreement in Subjectivity Analysis
Junyu Lu, Deyi Ji, Xuanyi Liu et al. (+5 more)
Large
language models for subjectivity analysis are typically trained with aggregated labels, which compress variations in human judgment into a single supervision signal. This paradigm overlooks the intrinsic uncertainty of low-agreement samples and often induces overconfident predictions, undermining reliability and generalization in complex subjective settings. In this work, we advocate uncertainty-aware subjectivity analysis, where models are expected to make predictions while expressing uncertainty that reflects human disagreement. To operationalize this perspective, we propose a two-phase Disagreement Perception and Uncertainty
Alignment (DPUA) framework. Specifically, DPUA jointly models label prediction, rationale generation, and uncertainty expression under an uncertainty-aware setting. In the disagreement perception phase, adaptive decoupled learning enhances the model's sensitivity to disagreement-related cues while preserving task performance. In the uncertainty
alignment phase, GRPO-based reward optimization further improves uncertainty-aware
reasoning and aligns the model's confidence expression with the human disagreement distribution. Experiments on three subjectivity analysis tasks show that DPUA preserves task performance while better aligning model uncertainty with human disagreement, mitigating overconfidence on boundary samples, and improving out-of-distribution generalization.
Read the full abstract ▾
All authors Junyu Lu, Deyi Ji, Xuanyi Liu, Lanyun Zhu, Bo Xu, Liang Yang, Xian-Sheng Hua, Hongfei Lin
Computation and Language
Phoenix-VL 1.5 Medium Technical Report
Team Phoenix: Arka Ray, Askar Ali Mohamed Jawad, Biondi Lee et al. (+26 more)
We introduce Phoenix-VL 1.5 Medium, a 123B-parameter natively
multimodal and
multilingual foundation model, adapted to regional
languages and the Singapore context. Developed as a sovereign AI asset, it demonstrates that deep domain adaptation can be achieved with minimal degradation to broad-spectrum intelligence and
alignment. Continued pretraining was performed on Mistral Medium 3.1 using a localized 1-trillion
tokens multimodal corpus, followed by a 250-billion
tokens long-context extension phase. Subsequent post-training incorporated a novel human-annotated Singapore
multimodal dataset and curated textual corpus on Singapore culture, knowledge, and legislation, totaling 22-billion
tokens. An additional 5 billion
tokens of model
alignment was performed through Online Direct Preference Optimization. Phoenix-VL 1.5 Medium achieves state-of-the-art performance for its size on Singapore
multimodal, legal, and government policy
benchmarks while remaining globally competitive on general
multimodal intelligence,
multilingual, and STEM
benchmarks. We also introduce a novel evaluation suite encompassing localized knowledge
benchmarks and an institutionally aligned model behavior and safety framework. We report the data curation principles, training methodology, and highlight
benchmark and inference performance.
Read the full abstract ▾
All authors Team Phoenix: Arka Ray, Askar Ali Mohamed Jawad, Biondi Lee, Elijah Seah, Eva Lim, Fiona Teo, Grace Toh, Guang Xiang Teo, Jun En Tan, Jia Hui Bong, Jiale Wang, Jonathan Ng, Justin Tan, Kai Zhe Yew, Matthew Ong, Shun Yi Yeo, Wen Jett Lam, Wen Xiu Tan, Ze Yu Zhang, Gee Wah Ng, Chee Wee Ang, Mistral AI: Adrien Sadé, Guillaume Kunsch, Jia Sin Loh, Nicolas Schuhl, Rupert Menneer, Umar Jamil, Vincent Maladière, Yimu Pan
Computation and Language
Not All Proofs Are Equal: Evaluating LLM Proof Quality Beyond Correctness
Ivo Petrov, Jasper Dekoninck, Dimitar I. Dimitrov et al. (+1 more)
Large
language models (LLMs) have become capable mathematical problem-solvers, often producing correct proofs for challenging problems. However, correctness alone is not sufficient: mathematical proofs should also be clear, insightful, contain transferable knowledge, and be generally useful to the reader. While proof quality is subjective and depends on the reader and context, many of its components can be made concrete. In this work, we identify such components and introduce ProofRank, a
benchmark curated from challenging competitions that measures several scalable proxies of proof quality: (i) conciseness - quantifying whether proofs avoid unnecessary steps; (ii) computational ease - the extent to which a proof relies on tedious calculations; (iii) cognitive simplicity - how accessible the used proof techniques are; (iv) diversity - how varied a model's proofs for a single problem are; and (v) adaptivity - how well a model can follow a specified technique. We find proof quality varies substantially across models along dimensions that correctness largely fails to capture. Every proof-level proxy can be steered through prompting, but often at a moderate correctness cost. ProofRank thus captures important aspects of mathematical proofs beyond correctness, and indicates the need for models that produce proofs which are not only correct but also valuable.
Read the full abstract ▾
All authors Ivo Petrov, Jasper Dekoninck, Dimitar I. Dimitrov, Martin Vechev
Computation and Language · Cross-listed from cs.DB
Toward Multi-Database Query Reasoning for Text2Cypher
Makbule Gulcin Ozsoy
Large
language models have significantly improved natural
language interfaces to databases by translating user questions into executable queries. In particular, Text2Cypher focuses on generating Cypher queries for graph databases, enabling users to access graph data without query
language expertise. Most existing Text2Cypher systems assume a single preselected graph database, where queries are generated over a known schema. However, real-world systems are often distributed across multiple independent graph databases organized by domain or system boundaries, where relevant information may span multiple sources. To address this limitation, we propose a shift from single-database query generation to multi-database query
reasoning. Instead of assuming a fixed execution context, the system must reason about (i) relevant databases, (ii) how to decompose a question across them, and (iii) how to integrate partial results. We formalize this setting through a three-phase roadmap: database routing, multi-database decomposition, and heterogeneous query
reasoning across database types and query
languages. This work provides a structured formulation of multi-database
reasoning for Text2Cypher and identifies challenges in source selection, query decomposition, and result integration, aiming to support more realistic and scalable natural
language interfaces to graph databases.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
How Mobile World Model Guides GUI Agents?
Weikai Xu, Kun Huang, Yunren Feng et al. (+10 more)
Recent advances in
vision-
language models have enabled mobile GUI
agents to perceive visual interfaces and execute user instructions, but reliable prediction of action consequences remains critical for long-horizon and high-risk interactions. Existing mobile world models provide either text-based or image-based future states, yet it remains unclear which representation is useful, whether generated rollouts can replace real environments, and how test-time guidance helps
agents of different strengths. To answer the above questions, we filter and annotate mobile world-model data, then train world models across four modalities: delta text, full text,
diffusion-based images, and renderable code. These models achieve SoTA performance on both MobileWorldBench and Code2WorldBench. Furthermore, by evaluating their downstream utility on AITZ, AndroidControl, and AndroidWorld, we obtain three findings. First, renderable code reconstruction achieves high in-distribution fidelity and provides effective
multimodal supervision for data construction, while text-based feedback is more robust for online out-of-distribution (OOD) execution. Second, world-model-generated trajectories can provide transferable interaction experience in the training process and improve
agents' end-to-end task performance, although these data do not preserve the original distribution. Last, for overconfident mobile
agents with low action
entropy, posterior self-reflection provides limited gains, suggesting that world models are more effective as prior perception or training supervision than as universal post-hoc verifiers.
Read the full abstract ▾
All authors Weikai Xu, Kun Huang, Yunren Feng, Jiaxing Li, Yuhan Chen, Yuxuan Liu, Zhizheng Jiang, Heng Qu, Pengzhi Gao, Wei Liu, Jian Luan, Xiaolin Hu, Bo An
Computation and Language
An Annotation Scheme and Classifier for Personal Facts in Dialogue
Konstantin Zaitsev
The advancement of Large
Language Models (LLMs) has enabled their application in personalized dialogue systems. We present an extended annotation scheme for personal fact classification that addresses limitations in existing approaches, particularly PeaCoK. Our scheme introduces new categories (Demographics, Possessions) and attributes (Duration, Validity, Followup) that enable structured storage, quality filtering, and identification of facts suitable for dialogue continuation. We manually annotated 2,779 facts from Multi-Session Chat and trained a multi-head classifier based on
transformer encoders. Combined with the Gemma-300M encoder, the classifier achieves $81.6 \pm 2.6$% macro F1, outperforming all few-shot LLM baselines (best: GPT-5.4-mini, 72.92%) by nearly 9 percentage points while requiring substantially fewer computational resources. Error analysis reveals persistent challenges in semantic boundary disambiguation, temporal aspect interpretation, and pragmatic
reasoning for followup assessment. The
dataset\footnotemark[1] and classifier\footnotemark[2] are publicly available.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
PowerStep: Memory-Efficient Adaptive Optimization via $\ell_p$-Norm Steepest Descent
Yao Lu, Dengdong Fan, Shixun Zhang et al. (+1 more)
Adaptive optimizers such as Adam are standard for training
Transformers, but storing gradient first and second moments incurs substantial
memory overhead. We introduce PowerStep, a
memory-efficient optimizer that achieves coordinate-wise adaptivity without storing second-moment statistics. Motivated by $\ell_p$-norm steepest descent, PowerStep applies a signed-power transform directly to one momentum buffer. We establish a finite-horizon stationarity bound for exact, unregularized updates, with an $O(1/\sqrt{T})$ term and a noise-dependent residual. Experiments on
Transformers from 124M to 235B parameters show competitive validation quality while halving $\texttt{fp32}$ optimizer-state
memory relative to AdamW. Combined with uniform $\texttt{int8}$ quantization, PowerStep remains numerically stable and reduces optimizer-state
memory by $\sim8\times$ compared to $\texttt{fp32}$ AdamW. PowerStep thus provides a simple,
memory-efficient alternative for large-scale training.
Read the full abstract ▾
All authors Yao Lu, Dengdong Fan, Shixun Zhang, Yonghong Tian
Computation and Language
ANCHOR: Abductive Network Construction with Hierarchical Orchestration for Reliable Probability Inference in Large Language Models
Wentao Qiu, Guanran Luo, Zhongquan Jian et al. (+3 more)
A central challenge in large-scale decision-making under incomplete information is estimating reliable probabilities. Recent approaches use Large
Language Models (LLMs) to generate explanatory factors and coarse-grained probability estimates, which are then refined by a Naïve Bayes model over factor combinations. However, sparse factor spaces often yield “unknown” predictions, while expanding factors increases noise and spurious correlations, weakening conditional independence and degrading reliability. To address these limitations, we propose Anchor, an aggregated
Bayesian inference framework over a hierarchical factor space. It constructs dense factor hierarchies through iterative generation and clustering, maps contexts via hierarchical
retrieval and refinement, and augments Naïve Bayes with a Causal
Bayesian Network to model latent factor dependencies. Experiments show that Anchor markedly reduces “unknown” predictions and produces more reliable probability estimates than direct LLM baselines, achieving state-of-the-art performance while significantly reducing time and
token overhead.
Read the full abstract ▾
All authors Wentao Qiu, Guanran Luo, Zhongquan Jian, Jingqi Gao, Meihong Wang, Qingqiang Wu