Computation and Language
SEAD: Competence-Aware On-Policy Distillation via Entropy-Guided Supervision
Chia-Hsuan Lee, Zelei Cheng, Yu Wang et al. (+4 more)
On-policy
distillation (OPD) has a property absent in offline
distillation and RL: teacher supervision quality depends on student competence. Incoherent rollouts yield noisy gradients; already-mastered
tokens yield redundant ones. This creates waste at three scales (
tokens, training phases, and
prompts) yet existing methods supervise uniformly. We introduce SEAD, which uses
entropy as a unified probe of this competence-dependent degradation at three scales: (1) joint teacher-student
entropy partitions
tokens into zones receiving tailored divergences or zero gradient (approx. 50% skipped); (2) a cosine schedule anneals from forward to reverse KL as competence grows; (3) a competence-gated curriculum introduces
prompts easy-to-hard. These components are symbiotically necessary:
token selection requires coherent rollouts (curriculum), annealing requires monotonic improvement (also curriculum). On OLMo-3 (7B to 32B), SEAD achieves +4.8 avg accuracy over vanilla OPD across six math
benchmarks, with ablations confirming super-additive interactions.
Read the full abstract ▾
All authors Chia-Hsuan Lee, Zelei Cheng, Yu Wang, Renkun Ni, Sambit Sahu, Shi-Xiong Zhang, William Campbell
Computation and Language
Depth-Staggered Fibonacci Spacing for Sparse Attention: Static Schedules Beat Learned Dilation and Extrapolate Where Dense Attention Fails
Chad A. Capps
We study sparse self-
attention in which each query attends to a dense local window plus a set of Fibonacci-spaced offsets, with a per-layer scalar alpha that compresses or expands the spacing. Across 21
language models trained under one matched recipe (60M parameters, 512 hidden, 16 layers, 426M
tokens), we compare four ways of setting alpha across depth: fixed, per-layer learned, a static linear stagger, and a coprime (anti-gridding) reassignment of that stagger, together with a reach-matched power-of-2 control. Three results stand out. First, a static per-layer stagger improves perplexity over both fixed and learned alpha, and the gain is base-agnostic: applying the same stagger to a power-of-2 base lifts it above fixed Fibonacci and to parity with learned Fibonacci
attention. Second, learning per layer is inert: it does not beat the static schedule and costs roughly five times the inference latency. Third, and most consequential, all sparse variants extrapolate to four times their training length with little or no degradation, whereas a recipe-matched dense baseline collapses (perplexity rises by 201% at 4x length); we attribute this to fixed-offset
attention only ever querying relative positions seen during training. We also report two honest negatives: at training length the best sparse model has about 26% higher perplexity than the dense baseline, and the staggering gain is uniform across context positions rather than concentrated at long range.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Matteo Farina, Vishaal Udandarao, Thao Nguyen et al. (+33 more)
Building performant
Vision-
Language Models (VLMs) requires carefully curating large-scale training
datasets, yet the community lacks systematic
benchmarks for evaluating such curation strategies. We introduce DataComp for VLMs (DCVLM), a
benchmark for controlled data-centric experiments to improve VLM training. As part of DCVLM, we collect 160
datasets spanning four data types – image-caption pairs,
multimodal interleaved documents, text-only, and instruction-tuning data – into a corpus of 6T
multimodal tokens. DCVLM allows participants to test curation strategies (filtering, mixing, formatting, sampling) across 1B-8B models and 6.25B-200B
token budgets. Models are then evaluated on a carefully selected suite of up to 52 downstream
benchmarks across 9 domains. We conduct extensive experiments on DCVLM and find that data mixing, not filtering, is key to a high-quality training
dataset: instruction-heavy mixtures scale better than caption-heavy ones, with gains widening at larger scales. The resulting
dataset, DCVLM-Baseline, enables training an 8B VLM to 63.6% accuracy on our 33-task core suite with 200B training
tokens. Compared to FineVision, the state-of-the-art open VLM training
dataset, this represents an improvement of +5.4pp. DCVLM and all accompanying artifacts will be made publicly available at https://www.datacomp.ai/dcvlm/.
Read the full abstract ▾
All authors Matteo Farina, Vishaal Udandarao, Thao Nguyen, Selim Kuzucu, Maximilian Böther, Andreas Hochlehnert, Adhiraj Ghosh, Marianna Nezhurina, Karsten Roth, Joschka Struber, Yuhui Zhang, Sebastian Dziadzio, Elaine Sui, Soumya Jahagirdar, Dhruba Ghosh, Hasan Hammoud, Thomas De Min, Simone Caldarella, Jehanzeb Mirza, Sedrick Keh, Mehdi Cherti, Hilde Kuehne, Bernt Schiele, Serena Yeung-Levy, Muhammad Ferjad Naeem, Federico Tombari, Ana Klimovic, Elisa Ricci, Matthias Bethge, Sewoong Oh, Ameya Prabhu, Alessio Tonioni, Jenia Jitsev, Massimiliano Mancini, Ludwig Schmidt, Nikhil Parthasarathy
Computation and Language
Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution
Kevin Der, Harish Kamath and Ben Thompson
Sparse autoencoders (SAEs) have become a useful tool for extracting interpretable features in
language models. However, standard SAE architectures operate on individual
token activations, meaning that the number of active features scales linearly with context length, and studying long model transcripts becomes difficult. We introduce turn-averaged SAEs, which represent a single Human or Assistant turn with a fixed number of features by learning to reconstruct the average model activation across the turn. We find that turn-averaged features describe a single turn's high-level characteristics more completely than per-
token features when judged by an LLM. We also demonstrate that turn-averaged SAEs greatly simplify common downstream uses of SAEs like attribution graphs. Broadly, turn-averaged SAEs make interpretability techniques practical at long context lengths.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CY
Who Plays Which Role When? Communication Role Dynamics for Peer Recognition and Team Performance Prediction
Yifan Song, Wenxuan Wendy Shi, Brian P Bailey et al. (+1 more)
Team roles offer an interpretable lens on collaboration, yet computational studies of roles often rely on domain-specific personas or data-driven clustering rather than theory-grounded taxonomies. We operationalize a taxonomy of eight communication roles grounded in education literature and annotate a corpus of 6,307 Slack messages from 55 students across 18 teams in a semester-long computer science course project. We evaluate whether LLMs can approximate expert labels, enabling scalable, taxonomy-driven role annotation. Using these role labels, we characterize role dynamics over teams' lifecycles, finding that different roles peak at different moments and that students enact a more diverse set of roles as projects progress. To evaluate the utility of our role constructs, we use them to predict peer recognition, outperforming lexical, conversational, and LLM-prompting baselines. To assess generalizability beyond the educational context, we apply the same role constructs to a public
dataset (DeliData) to predict team performance improvement after deliberation, again exceeding prior performance.
Read the full abstract ▾
All authors Yifan Song, Wenxuan Wendy Shi, Brian P Bailey, Tal August
Computation and Language
Legal Domain Adaptation of Modern BERT Models
Dominik Stammbach and Peter Henderson
We investigate domain adaptation of modern BERT models in the legal domain. We further pre-train ModernBERT on all US court opinions using the masked
language modeling objective. Although ModernBERT has been trained on roughly 500x more data than original BERT, we still find that this model benefits from further pre-training and domain adaptation in the legal domain: we report significant improvements compared to vanilla ModernBERT on all
datasets connected to US court opinions. We find gains similar to those reported in early work on domain adaptation of BERT-like models. However, from scratch pre-training does not match the performance of further pre-training an existing ModernBERT checkpoint in our experiments. The resulting models are capable of processing sequences up to 8,192
tokens, and can be used to compute meaningful
embeddings of legal passages, or could quickly rerank hundreds of legal passages for a given search query. We release all model checkpoints publicly.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.MM
A Good Talk Does not Look Like a Summary, It Teaches You! Measuring Takeaways from Paper-to-Video Talks
Ishani Mondal, Aparna Garimella, Ananya Sai et al. (+2 more)
Automatically generated
videos from scientific papers are increasingly used for education and research dissemination. However, existing evaluation metrics mainly measure visual quality or whether key points from the paper appear in the
video without assessing whether the
video actually helps viewers understand the ideas. We introduce EffectivePresentationScorer, a framework for evaluating the instructional quality of scientific presentation
videos. It checks whether a
video explains the main ideas clearly, introduces needed background concepts, and connects technical details to the main contribution of the paper. When we apply EffectivePresentationScorer to the existing paper-to-
video generation systems, we find that generated
videos mention the correct topics and follow the structure of the paper but fail to explain prerequisite concepts or clarify why the method works. These failures are often ignored by existing
video evaluation metrics, which focus on content presence rather than explanatory quality.
Read the full abstract ▾
All authors Ishani Mondal, Aparna Garimella, Ananya Sai, Pannaga Shivaswamy, Jordan Boyd-Graber
Computation and Language
A French OSCE Dialogue Dataset and Controllable Virtual Patient System for Clinical Training
Doria Bonzi, Tom Bourgeade, Fabrice Lefèvre et al. (+1 more)
The clinical and communication skills of medical students are commonly assessed through Objective Structured Clinical Examinations (OSCEs), which consist of brief scenario-driven simulations of doctor-patient interactions. However, training is often limited by the low availability of human standardized patients, motivating the development of realistic virtual patients (VPs). To address this gap, we introduce a French OSCE dialogue
dataset comprising 240 student-patient training interactions. We build upon it a controllable LLM-based pipeline to generate synthetic OSCE dialogues. The pipeline integrates modular components, such as
retrieval-based grounding and a reflection loop, to ensure patient fidelity, coherence, and realism. Additionally, we propose a multi-level evaluation framework assessing patient simulation quality, student performance, and linguistic quality, using an LLM-as-a-Judge approach. Experiments suggest that controllability modules generally improve patient fidelity and student evaluation consistency. Finally, we implement an interactive prototype in which students can practice with a VP and receive automatic feedback.
Read the full abstract ▾
All authors Doria Bonzi, Tom Bourgeade, Fabrice Lefèvre, Irina Illina
Computation and Language
Developmental Trajectories of Situation Modeling and Mentalizing in Transformer Language Models
Pamela D. Rivière, Cameron Jones and Sean Trott
Recent work suggests that Large
Language Models (LLMs) are sensitive to the belief states of
agents described by text, as measured by the false belief task (FBT), yet persistent concerns of construct validity remain. We adopt a **developmental perspective**, tracing the pattern of mental state
reasoning behavior – and likely **preconditions** for this behavior – across multiple training stages in the Olmo2 and Pythia
language model suites. We find that above-chance FBT performance depends both on model size and sufficient training volume, emerges relatively late in pretraining, and is most improved by post-training interventions (SFT, DPO) in the condition most diagnostic of mentalizing (False Belief, Implicit). However, FBT performance is fragile: consistent with past work, the use of non-factive verbs (e.g., thinks) increases false belief attributions even in the True Belief condition. To contextualize these findings, we track the emergence of **situation modeling**: the ability to report on basic factual properties of a described scene. Situation modeling accuracy generally precedes and exceeds FBT accuracy, yet situational representations also prove surprisingly incoherent in certain respects: when asked about the knowledge states of the Antagonist
agent – who always knows the item's true location – Olmo2 13b is consistently influenced both by the Target
agent's knowledge state and the presence of non-factive verbs. Together, these results suggest that larger, sufficiently trained models build partially coherent situation models in a developmentally appropriate sequence, yet display surprising fragility – highlighting the value of developmental and stress-testing approaches for evaluating LLM capabilities.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty
Xiao Song, Haonan Qin, Zhaoxu Zhang et al. (+3 more)
Large
vision-
language models (LVLMs) are increasingly used for clinical image understanding, yet they remain vulnerable to
hallucinations–producing textual findings or attributes not supported by the image. We present a
vision-traceable
hallucination detection framework that audits arbitrary LVLM responses via visual evidence grounding, requiring neither modification nor internal access to the hidden states of LVLMs. Given an LVLM response, we extract visually verifiable entities and use a medical-domain-adapted Qwen-VL grounding verifier to localize each entity on the input image. To enhance the robustness of our detection method, we introduce a counterfactual entity perturbation method and estimate visual evidence uncertainty by contrasting factual and counterfactual grounding results. Specifically, we compute an entity-level uncertainty score from the positive confidence, counterfactual confidence, and their grounding overlap for binary
hallucination decision-making. Experiments on multiple medical imaging modalities and LVLM backbones demonstrate that our method consistently improves
hallucination detection performance over recent baselines, while providing interpretable localization evidence and strong cross-model transferability. Code and
dataset are available at https://github.com/Agentic-CliniAI/CounterVHD.
Read the full abstract ▾
All authors Xiao Song, Haonan Qin, Zhaoxu Zhang, Jiong Zhang, Yuqi Fang, Caifeng Shan
Computation and Language · Cross-listed from cs.AI
GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes
Amit Parekh, Sabrina McCallum, Kareem Al-Hasan et al. (+3 more)
Multimodal models are increasingly deployed to solve tasks collaboratively with humans or other artificial
agents. While existing
benchmarks show that they possess the fundamental capabilities, the various conditions that coincide when collaborating—time pressure, information asymmetry, and imperfect communication—have traditionally been studied in isolation. To address this gap, we introduce GPTNT, a
benchmark built on the cooperative
video game Keep Talking and Nobody Explodes, in which two
agents must coordinate to defuse procedurally generated bomb puzzles against a live countdown. One
agent has access to the bomb but not the instructions for defusing it; the other holds the instructions but cannot see or manipulate the bomb. Neither
agent can succeed alone: the task requires contributions from both, and is solvable only through effective, efficient communication. We remove turn-taking proxies or simplifications, instead requiring
agents to act asynchronously and communicate in real time. GPTNT is designed to expose how models collaborate versus how they perform alone: the instruction manual, the partner, or both, can optionally be withheld to surface what a model has memorised versus what it derives in the moment. We demonstrate that GPTNT poses a considerable challenge to the state-of-the-art: not one of the closed- and open-source models we test defuses a single bomb in real time, a bar that human players clear. In a range of controlled experiments, we explore where capabilities break down, identifying critical weaknesses in state tracking, efficient acting within the time budget, handling ambiguity, and error recovery. Since it runs on the real game, GPTNT benefits from procedural generation and inherits a living modding community: as models improve, the
benchmark can be evolved to remain challenging, rather than being solved once and retired.
Read the full abstract ▾
All authors Amit Parekh, Sabrina McCallum, Kareem Al-Hasan, Malvina Nikandrou, Alessandro Suglia, Ioannis Konstas
Computation and Language · Cross-listed from cs.LG
An Agentic AI Pipeline for Appliance-Level Energy Anomaly Detection and LLM-Driven Recommendations
Dihia Falouz, Aida Douaibia, Amine Bechar et al. (+3 more)
Appliance-level energy monitoring in office buildings produces noisy alerts that non-expert facility managers struggle to use. This paper proposes an end-to-end agentic pipeline that combines deep time-series
forecasting, variational anomaly detection, and LLM-based
reasoning to generate prioritized, actionable maintenance recommendations. The system tracks seven office appliances using a hybrid Singular Spectrum Analysis (SSA) and Long Short-Term
Memory (LSTM)
forecasting model, and applies a per-appliance LSTM Variational Autoencoder (VAE) with
attention to flag abnormal daily consumption episodes. A three-stage LangChain pipeline begins with a Context
Agent that always retrieves three core RAG sources (model reliability, hourly baseline, and expert knowledge) and conditionally adds up to three more (forecast context, anomaly history, global baseline) based on event characteristics, capped at eight
reasoning steps. A Diagnosis
Agent converts the evidence into a structured JSON diagnosis, and a Report
Agent renders a human-readable narrative. A reflective
memory layer incorporates operator feedback. The dashboard shows real-time 30-minute forecasts, intraday consumption, the previous day anomaly report, and a feedback form. We evaluate the
forecasting model, anomaly detector with appliance-specific thresholds, and LLM
reasoning on a 16-scenario
benchmark including sustained and transient spikes, unexpected shutdowns, and systemic events, comparing five LLM backends under static vs. dynamic
retrieval. Dynamic
retrieval matches full static
retrieval across all backends while cutting average context from six to three-six sources per event. The best backend scores 90.4/100 with a 100% pass rate at a 70-point threshold, and a fully local 7B-parameter model passes all 16 scenarios.
Read the full abstract ▾
All authors Dihia Falouz, Aida Douaibia, Amine Bechar, Youssef Elmir, Abbes Amira, Adel Oulefki
Computation and Language
Extracting Knowledge from an Arabic-English Machine-Readable Dictionary Using Information Extraction
Diaa M. Fayed, Aly A. Fahmy, Mohsen A. Rashwan et al. (+1 more)
Natural
language processing (NLP) applications need large and rich amount of linguistic knowledge. Furthermore, electronic
language sources such as dictionaries, encyclopedia, and corpora became available. So, automatic methods are emerged to extract lexical information from those sources to overcome the knowledge acquisition bottleneck. We presented a method to automatically extract lexical information from a machine-readable version of the Arabic-English Al-Mawrid dictionary. We used n-gram analysis and key-word-in-context (KWIC) analysis to discover lexical patterns that manifest morphologic, syntactic, or semantic information. Then, we used hand-crafted rule-based information extraction to extract that information. Furthermore, we used punctuation marks and some heuristics to extract a set of synonyms in a subentry. This study registered high precision for all types of information, high recall for synonyms, and low recall for the other information. The study also showed that the Al-Mawrid has significant amount of derivations (morphologic information) and synonyms, domain labels, and hyponym/hypernym relations (semantic information).
Read the full abstract ▾
All authors Diaa M. Fayed, Aly A. Fahmy, Mohsen A. Rashwan, Wafaa K. Fayed
Computation and Language · Cross-listed from cs.SD
LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features
Jonghyeon Park, Olivier Jiyoun Jung and Myungwoo Oh
Early detection of dementia enables timely intervention, and reflecting cognitive impairment, spontaneous
speech offers a non-invasive screening modality. Conventional approaches often focus on a single representational dimension – such as acoustic descriptors, pause modeling, automatic
speech recognition (ASR) transcripts, or
multimodal fusion – limiting integrative
reasoning across heterogeneous cognitive symptoms. We propose a low-rank adaptation (LoRA)-tuned large
language model (LLM) that performs structured multi-view
reasoning over four complementary
speech-derived signals: ASR transcripts with pause markers, discourse-level topic cues, temporal fluency statistics, and phonological sequences. These cues are encoded within a unified
prompt, enabling a single LLM to learn a coherent decision function without modality-specific encoders or late-stage fusion. On ADReSSo, our best model achieves an F1-score of 90.14%, and ablation confirms the complementary contribution of each view.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Towards Automating Scientific Review with Google's Paper Assistant Tool
Rajesh Jayaram, Drew Tyler, David Woodruff et al. (+4 more)
Artificial intelligence is driving a revolution in scientific discovery, accelerating everything from hypothesis generation to mathematical
theorem proving. However, this rapid acceleration is creating a systemic challenge: traditional human peer review cannot scale to match the influx of AI-assisted science. Ultimately, to resolve this tension, we must also deploy AI to accelerate the verification and review process itself. To frame the discussion around this transition, we propose a taxonomy consisting of four progressive levels of AI-human collaboration in scientific evaluation, and discuss various trade-offs involved with each. As a step toward this future, we introduce the Paper Assistant Tool (PAT), an agentic AI framework built for deep scientific review and verification. PAT ingests full scientific manuscripts and produces a comprehensive evaluation, checking theoretical results, validating experiments, suggesting improvements, and identifying potential flaws. By utilizing inference scaling techniques, PAT is able to identify deeper issues than a single model call alone, achieving a 34% improvement over zero-shot recall on mathematical errors in the SPOT
benchmark. Pilot deployments of PAT as a pre-submission tool for authors at two major Computer Science conferences – STOC and ICML – demonstrate its ability to identify critical errors and suggest substantive improvements to research papers. By catching errors early, PAT eases the cognitive burden placed on referees, while preserving their control over the outcomes of the review process.
Read the full abstract ▾
All authors Rajesh Jayaram, Drew Tyler, David Woodruff, Corinna Cortes, Yossi Matias, Vahab Mirrokni, Vincent Cohen-Addad
Computation and Language
Vision-Default, Prior-Override: Causal Mechanisms of Perception-Knowledge Conflict in Vision-Language Models
Niclas Lietzow, Danielle Bitterman, Carsten Eickhoff et al. (+2 more)
Vision-
language models must reconcile visual evidence with memorized world knowledge when the two conflict. How they resolve this conflict shapes the reliability of
multimodal systems, yet prior work characterizes it behaviorally without a component-level causal account. We combine activation patching across three granularities (residual stream,
attention heads, and MLP sublayers) with model-component ablation studies and mechanistic analysis. Across three VLM families, we find that visual grounding emerges by default, whereas prior grounding depends on a small set of causally necessary
attention heads (2.5-4.8%) concentrated in the second half of the network. These heads enable answers from stored world knowledge (e.g., "red" for a strawberry) despite conflicting visual input. Ablating them flips predictions from knowledge-grounded to visually grounded answers in 68-96% of cases under prior-knowledge
prompts, but changes only 0.8-7.5% of visually grounded predictions, establishing an asymmetric causal structure. The identified heads decompose into routing heads, which modulate information flow, and writing heads, which directly project answer
tokens into the residual stream. This structure is consistent across model families and scales, revealing a sparse causal
circuit underlying perception-knowledge conflict in VLMs.
Read the full abstract ▾
All authors Niclas Lietzow, Danielle Bitterman, Carsten Eickhoff, William Rudman, Michal Golovanevsky
Computation and Language · Cross-listed from eess.AS
HPRO: Hierarchical Progressive Reward Optimization via Preference Extraction for Emotional Text-to-Speech
Sihang Nie, Xiaofen Xing, Rui Xing et al. (+5 more)
Recently, Large
Language Model (LLM)-based Text-to-
Speech (TTS) models have achieved remarkable naturalness. However, the standard Supervised Fine-Tuning paradigm often converges to statistically averaged prosody, limiting emotional expressiveness. While preference-driven optimization offers a promising alternative, existing approaches suffer from two structural mismatches: information conflict, where content and
emotion in a shared latent space produce conflicting gradients, leading to reward hacking and semantic degradation; and scale gap, where sparse sentence-level rewards struggle to guide dense frame-level generation. To overcome these challenges, we propose HPRO, a hierarchical progressive reward optimization framework. Within HPRO, we introduce the HD-Emo codec as a novel differentiable reward model to mitigate the information conflict. It extracts
speech into distinct content and style preference
tokens, structurally isolating emotional optimization from semantic content. Building upon this structured preference space, HPRO bridges the scale gap by progressively aligning frame-, word- and sentence-level objectives. Experiments demonstrate that HPRO significantly enhances emotional expressiveness, while effectively preserving linguistic intelligibility. The code and audio samples are publicly available at https://xxh333.github.io/hpro-demo/.
Read the full abstract ▾
All authors Sihang Nie, Xiaofen Xing, Rui Xing, Haoming Li, Ruitong Xiao, Jingyuan Xing, Baiji Liu, Xiangmin Xu
Computation and Language
Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction
Chenguang Wang, Ming Li, Xinyue Zeng et al. (+4 more)
Predicting human item difficulty is central to educational assessment, where reliable estimates support
fairness and effective test construction. Existing methods often depend on costly human calibration or item-level textual representations, providing limited evidence about the cognitive processes that make items difficult. We argue that difficulty should be viewed not only as a property of item text, but also as an observable consequence of the problem-solving burden an item induces. Large
Reasoning Models (LRMs) offer scalable process evidence through
reasoning traces, but such evidence must be structured to support interpretable modeling. To this end, we introduce Epi2Diff (Episode to Difficulty), a framework that maps LRM
reasoning traces into cognitively grounded episode sequences. These episodes group trace segments into functional problem-solving states, enabling difficulty to be modeled through
reasoning scale, effort allocation, and state transitions. Epi2Diff extracts compact episode-dynamic features and combines them with semantic item representations for human difficulty prediction. Experiments on four real-world human difficulty
datasets show that Epi2Diff consistently outperforms strong baselines, including fine-tuned small
language models, LLM in-context learning, and supervised LLM adaptation. On SAT-derived classification
benchmarks, Epi2Diff achieves an 8.1% average relative gain over supervised LLM fine-tuning baselines. Further analyses show that harder items induce more effortful, iterative, and implementation-centered episode dynamics, rather than merely longer responses. These results demonstrate that cognitive episodes in LRM
reasoning traces provide a predictive and interpretable process representation for human item difficulty, offering a new lens for educational measurement with
reasoning models.
Read the full abstract ▾
All authors Chenguang Wang, Ming Li, Xinyue Zeng, Zhuochun Li, Hong Jiao, Tianyi Zhou, Dawei Zhou
Computation and Language
From Tokens to States: LLMs as a Special Case of World Models and the Continuous Path Beyond
Paul Dubois
The AI community has framed the relationship between large
language models (LLMs) and world models as a dichotomy: LLMs predict
tokens; world models simulate reality. Yann LeCun argues in 2022 that reaching general intelligence requires abandoning autoregressive
token prediction in favour of latent-space architectures. This framing is unnecessarily binary. Two claims will be defended. First, LLMs are a degenerate special case of world models: the state space is the set of all
token sequences, the only action is appending one
token, and world models are therefore a strict generalisation of LLMs, not a replacement. Second, there is a natural continuous spectrum from NTP to JEPA, with multi-
token prediction, future-summary prediction, and next-latent prediction as intermediate stations already populated by current research. Moving along this spectrum relaxes the LLM constraints one by one. It also progressively surrenders the two practical advantages that make LLMs trainable at scale: internet-scale self-supervised data, and a
transformer architecture co-designed for discrete
token prediction. Both are examined as open research questions: the data question (the cliff from self-supervised text to instrumented action-labelled environments) and the architecture question (whether the
transformer generalises to continuous-state prediction, or whether a new primitive is needed).
Read the full abstract ▾
Computation and Language
Mechanism-Aligned Observability for Training Instability of Large Language Models
Ruixuan Huang, Yipei Wang, Wenyi Fang et al. (+5 more)
Frontier large
language model training consumes massive accelerator fleets over long wall-clock times, making stability failures extremely costly. However, global symptoms such as loss divergences often emerge only after the underlying fault has already affected the training dynamics, further increasing the sunk compute. In which internal representation can a training fault become persistently and interpretably observable before it appears in global training metrics? We propose a mechanism-aligned principle for training observability: identify the internal state into which a fault is persistently written, derive the structure induced by its mechanism, and construct monitors in representations aligned with the computation of the affected module. For FlashAttention, we show that biased backward errors accumulate into a persistent low-dimensional structure in weight updates, producing spectral concentration. For MoE routers, we characterize routing dynamics through per-
token routing behavior rather than parameter similarity, which may change without altering the routing function. Experiments show that the two monitors respond selectively to changes in their corresponding modules, before degradation becomes visible in global training metrics.
Read the full abstract ▾
All authors Ruixuan Huang, Yipei Wang, Wenyi Fang, Hantao Huang, Ansheng You, Shuai Wang, Fan Wu, Yang Zheng
Computation and Language · Cross-listed from cond-mat.dis-nn
Scaling limit of the Random Language Model
Eric De Giuli
We develop a quantitative theory of the Random
Language Model (RLM), an ensemble of
stochastic context-free grammars, in a scaling limit where the number of hidden symbols $N \to \infty$ while the grammar temperature $\tilde{\epsilon}_d \to 0$ at fixed $x = {\tilde\epsilon}_d \log N$. In this limit, the model admits a controlled description based on a large-deviation principle over rule-usage patterns. A semi-annealed approximation maps the problem to a class of Random Energy Models with nontrivial combinatorics. We show that the RLM exhibits a condensation transition at a critical value $x_c=1/8$, below which rule usage concentrates and
language statistics acquire a nontrivial dependence on corpus length. A second characteristic scale at $x=1/2$ marks the onset of
entropy reduction from its maximal value. Across these regimes, we derive explicit scaling laws for the number of distinct rules,
entropy, and related observables, identifying distinct scaling, saturation, and critical regimes controlled by the interplay of grammar size, corpus length, and temperature. The theory resolves previous ambiguities regarding the existence of a thermodynamic transition and explains the slow approach to the large-$N$ limit as a consequence of the dependence on $\log N$. It further provides a unified framework in which universal statistical properties of
language emerge from typical realizations of generative grammars, with implications for both natural
language statistics and the behavior of large
language models.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.DB
Single and Multi Truth Data Fusion using Large Language Models
Hira Beril Kucuk, Norman W Paton, Jiaoyan Chen et al. (+1 more)
Data fusion, also known as truth discovery, is a data integration problem that aims to determine the correct value or set of values for each attribute of an object when presented with potentially conflicting values from multiple sources. Data fusion tasks belong to two main categories: single-truth scenarios, where each attribute has only one correct value, and multi-truth scenarios, where multiple values can be valid simultaneously. This paper investigates the use of Large
Language Models (LLMs) in data fusion tasks for tabular data. Various prompting strategies, encompassing both single-truth and multi-truth scenarios, are investigated empirically. Domain-dependent, domain-independent, zero-shot and one-shot
prompts are evaluated on three different
benchmark datasets. Experimental results demonstrate that LLM-based approaches outperform traditional unsupervised truth discovery methods, such as DART and LTM, across all
datasets. The codebase of this study has been made publicly available on GitHub.
Read the full abstract ▾
All authors Hira Beril Kucuk, Norman W Paton, Jiaoyan Chen, Zhenyu Wu
Computation and Language
MultiHashFormer: Hash-based Generative Language Models
Huiyin Xue, Atsuki Yamaguchi and Nikolaos Aletras
Language models (LMs) represent
tokens using
embedding matrices that scale linearly with the vocabulary size. To constrain the parameter footprint, prior work proposes hashing many
tokens into a single vector within encoder-only models. While this offers parameter efficiency, many-to-one collisions prevent its use in causal LMs. In this paper, we propose MultiHashFormer, a new framework that allows hash-based autoregression. Each
token is represented as a unique hash signature, a short sequence of discrete hash IDs, generated by multiple independent hash functions. A Hash Encoder compresses this signature into a single latent vector for processing by a
Transformer decoder. Then, a Hash Decoder generates the hash signature of the next
token, which is then mapped back to text. We evaluate our approach at the 100M, 1B and 3B parameter scales, demonstrating that MultiHashFormer consistently outperforms standard
Transformer LMs across multiple
benchmarks. Furthermore, we show that our model handles
multilingual vocabulary expansion with a constant parameter footprint without any modifications.
Read the full abstract ▾
Computation and Language
Can LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QA
Sambaran Bandyopadhyay
LLM-as-a-Judge and self-evaluation pipelines implicitly assume that evaluation is easier than generation. We test this in a controlled in-context QA setting where a context passage is the sole information source and each model judges the answer it generated, removing the parametric-knowledge confound of open-domain comparisons. Across four
benchmarks (SQuAD 2.0, DROP, HotpotQA, MuSiQue) and two models, evaluation is not uniformly easier: generation accuracy exceeds self-evaluation on three of four, with multi-hop MuSiQue the exception.
Attention analysis reveals why: evaluation attends to context 3–5x less than generation does and barely reads the candidate answer. LoRA fine-tuning confirms the asymmetry is not a training artifact: generation fine-tuning induces over-acceptance and evaluation fine-tuning degrades generation. These findings challenge core assumptions in self-evaluation pipelines.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
DG^VoiC: Speaker Clustering for Fraud Investigation under Real Call-Centre Conditions
Muhammad Shakeel Akram, Amal Htait, Abdul Hamid Sadka et al. (+2 more)
Insurance fraud remains costly and operationally difficult, particularly in call-centre workflows where many customer interactions begin at FNOL. While recent fraud detection methods mainly rely on structured data, text, or images, repeated speaker identity across calls remains underused as an investigative signal. This paper presents DG^VoiC, a voice clustering framework for customer verification and cross-profile speaker linking on anonymised real call-centre audio. The approach combines sensitive information-aligned anonymisation,
speech-focused preprocessing, sliding-window speaker
embedding extraction, and cosine similarity based clustering to identify repeated speakers under real telephony conditions. The method was evaluated on 121 recordings, with a curated reference subset of 56 samples in 22 human-agreed speaker clusters. used for validation. The best configuration achieved 96% AMI, 95% ARI, 98% completeness, 100% homogeneity, and 99% V-measure. These results show that speaker clustering can provide a strong additional signal for fraud investigation by helping analysts verify speaker consistency and surface repeated voices across customers.
Read the full abstract ▾
All authors Muhammad Shakeel Akram, Amal Htait, Abdul Hamid Sadka, Emma Meisingseth, Karishma Jaitly
Computation and Language
A Tree-of-Thoughts Inspired Hybrid Approach for Legal Case Judgement Summarization using LLMs
Aniket Deroy, Kripabandhu Ghosh and Saptarshi Ghosh
In recent times, Large
Language Models (LLMs) are increasingly being used for legal case judgement summarization. Most prior works have tried traditional extractive and abstractive summarization of case judgements. However, hybrid or extractive-abstractive techniques have not been explored much. In this work, we propose a novel tree-of-thoughts inspired extractive-abstractive summarization approach for legal judgement summarization. We conduct experiments using two popular LLMs, DeepSeek and LLama, and compare among extractive, abstractive and extractive-abstractive summarization. Our experiments show that the proposed extractive-abstractive
prompt provides better summaries compared to other types of LLM
prompts.
Read the full abstract ▾
Computation and Language
The Signal-Coverage Matrix: Stratifying Type and Semantic Errors in Statement Autoformalization
Chengxiao Dai, Zhaokun Yan and Zhanhui Lin
Headline type-correctness (TC%) of LLM autoformalization has climbed from $\sim$53% to $\sim$76% in two years, yet this scalar conceals which errors each method resolves. We propose a signal-coverage
matrix that crosses the Lean elaborator (pass/fail) with a semantic-equivalence judgment (equivalent/not), sorting every output into one of four
cells: true success (TS), type-only (TO), semantic-only (SO), or both fail (BF). On ProofNet\# and MiniF2F-test with DeepSeek V4-Pro across Vanilla, Lean-Retry, Sample-Filter, and Stratified Autoformalization (SAF): (1) the +34 to +36 TS gain across the three elab-feedback methods is $\sim$64% type-stratum recovery, with SO flat on net (87.5% of original semantic errors rescued, 8 newly created). (2) The TO-to-TS rate is 23/61 for each method (Wilson 95% CI [26.6%, 50.3%]), and this stratum-level recovery rate predicts $\Delta$TS on held-out methods to within 2/186 and renders $\Delta$TC linear in the Vanilla elab-fail rate across six (model,
dataset)
cells ($R^2=0.96$). (3) The two judges disagree by 26 to 37 pp on elab-feedback outputs (vs. 7 pp on Vanilla), with 30 to 56% of symbolic-judge false negatives traceable to elaborator-forced rewrites. The persistent residual reduces to two gold-formalization errors. TC% gains should be credited by which
cell moved, not by the scalar alone.
Read the full abstract ▾
Computation and Language
Dialogue to Detection: A Multimodal Hybrid NLP Pipeline for Insurance Fraud Detection
Muhammad Shakeel Akram, Amal Htait, Abdul Hamid Sadka et al. (+2 more)
Insurance fraud imposes substantial financial losses and operational inefficiencies, raising premiums and impacting trust among legitimate policyholders. Early detection at FNOL remains a persistent challenge. Existing approaches rely largely on private, text-only
datasets, limiting progress on
multimodal methods that integrate linguistic, behavioural, and speaker-based indicators. We introduce a synthetic
multimodal framework that replicates FNOL conditions. It generates
agent-customer dialogue transcripts and two-speaker audios, performs ASR and diarisation. Downstream modules combine NER, regex-based feature extraction, LLM-RAG
retrieval, and speaker
embeddings in a rule-based risk score to flag narrative reuse, structural inconsistencies, and cross-case voice repetition while balancing sensitivity and false positives.
Dataset validation and component-level evaluations show stability and transfer potential, offering a reproducible baseline beyond text-only fraud detection.
Read the full abstract ▾
All authors Muhammad Shakeel Akram, Amal Htait, Abdul Hamid Sadka, Emma Meisingseth, Karishma Jaitly
Computation and Language
Stefan F. Schouten, Ilia Markov and Piek Vossen
We introduce a new, contextual,
multilingual dataset called ToxiREX: Toxic
REasoning in ConteXt. The
dataset consists of threads of Reddit comments and structured characterizations of what the comments imply, following a systematic toxic
reasoning schema developed in a previous paper. Using the schema allows us to capture and explain implicit and context-dependent toxicity, while supporting mappings to existing toxicity taxonomies. The
dataset includes comments in six
languages (English, Arabic, Turkish, Spanish, German, and Dutch), collected from posts connected to specific major events (e.g. the 2023 Turkey earthquakes; the Russian invasion of Ukraine). We describe the context-preserving preprocessing of the threads. We create a training set of 125 thousand comments which is annotated by a commercially available LLM, and a test set of just under three thousand comments that is annotated by native speakers. We show that apparent disagreements in the test set annotations often reflect defensible alternative interpretations rather than noise. Finally, we provide baseline results by prompting and fine-tuning
language models. To produce these results, we develop evaluation strategies for our hierarchical, schema-based predictions. While models perform better than random, there remains a lot of room for improvement, showing the task to be challenging. ToxiREX is the first
dataset to simultaneously incorporate multiple
languages, conversational context, and implicit toxicity, while using the toxic
reasoning schema for rich, structured annotations.
Dataset available at: https://github.com/cltl/toxirex
Read the full abstract ▾
Computation and Language
From Black-Box to Clinical Insight: A Multi-Stage Explainable Framework for Speech-Based Cognitive Impairment Detection
Yasaman Haghbin, Sina Rashidi, Ali Zolnour et al. (+6 more)
Speech-based cognitive impairment detection offers a noninvasive, accessible alternative to costly biomarker assays, yet
transformer-based models remain clinically uninterpretable. We propose a multi-stage explainability framework that translates black-box
transformer predictions into clinically grounded narratives by integrating SHapley Additive exPlanations (SHAP)-based
token attribution, theory-informed linguistic features, and a four-stage LLM
reasoning pipeline using LLaMA-3.1-70B-Instruct. Built on the SpeechCARE-Adaptive Gating Network
multimodal screening model (F1 = 72.11% on the NIA PREPARE
benchmark), the framework maps model outputs to four cognitive-linguistic dimensions, including lexical richness, syntactic complexity, and semantic coherence. Physician evaluation on 70 stratified English samples demonstrated strong
alignment with patient-level cognitive profiles, and a System Usability Scale score of 82/100 indicated high potential for clinical workflow integration.
Read the full abstract ▾
All authors Yasaman Haghbin, Sina Rashidi, Ali Zolnour, Fatemeh Taherinezhad, Ali Fartoot, Hossein Azadmaleki, James M Noble, Maryam Dadkhah, Maryam Zolnoori