Computation and Language
Lightning OPD 2.0: Mitigating Style Bias in Cross-Teacher On-Policy Distillation for Large Reasoning Models
Yecheng Wu, Song Han and Han Cai
On-policy
distillation (OPD) provides dense
token-level supervision from a teacher, but its effectiveness can depend on teacher consistency, meaning that the model providing OPD supervision should also have generated the demonstrations used to train the supervised fine-tuning (SFT) reference. However, this condition is frequently violated in practice when SFT data have mixed or unknown provenance or when different models are preferred for SFT data generation and subsequent
distillation. In such cross-teacher settings, even a stronger OPD teacher can yield little improvement over the SFT reference. We find that raw teacher–reference disagreement contains potentially useful context-specific teacher evidence as well as a recurring component associated with differences in wording, formatting, and
reasoning cadence. We introduce Lightning OPD 2.0 with cross-fitted style residualization, which uses rollout-level cross-fitting to estimate this recurring component as an operational proxy for style-
token bias and subtracts it before constructing the
token-level OPD update. Across mathematical
reasoning and code generation
benchmarks, Lightning OPD 2.0 consistently outperforms Lightning OPD in cross-teacher settings. Starting from Klear-Reasoner-8B-SFT, Lightning OPD 2.0 reaches 82.4% on AIME 2024 and 63.0% on LiveCodeBench v5. Together, these results establish Lightning OPD 2.0 as a practical approach to cross-teacher OPD, relaxing teacher consistency as a prerequisite and allowing the SFT data generator and
distillation teacher to be selected independently. Code will be released soon.
Read the full abstract ▾
Computation and Language
Beyond a Single Judge: The Evidence-Grounded, Social-Weighted Persona Panel for Generative UI Evaluation
Zheng Wu, Yibo Luo, Pu Zhang et al. (+2 more)
Generative UI (GenUI) lets large
language models synthesize a complete, renderable interface directly from a natural-
language instruction, but evaluating the quality of what they generate remains an open problem. Human evaluation is costly and rater-variant, while LLM-as-a-judge is scalable but reflects only a single implicit viewpoint, unable to capture how different populations of real users actually perceive the same interface. We propose the Evidence-Grounded, Social-Weighted Persona Panel (ESPP), a three-stage GenUI evaluation method in which a panel of psychologically diverse, evidence-grounded personas independently rates a screenshot, exchanges opinions under a trait-derived, semantically-gated bounded-confidence mechanism, and is aggregated via Delphi-inspired social weighting into a single judgment. ESPP tracks human judgment substantially more closely than a naive single-pass judge, raising Pearson $r$ from $0.716$ to $0.922$, and a
prompt-ensemble control recovers only about a third of this gap, isolating genuine persona and evidence grounding as the dominant source of improvement. Beyond this fidelity gain, retaining each panelist's individual rating further reveals that user subgroups agree on overall model rankings yet diverge sharply on specific rating dimensions, a structural disagreement a single homogeneous judge would systematically erase. The codes are available at https://github.com/Wuzheng02/ESPP.
Read the full abstract ▾
All authors Zheng Wu, Yibo Luo, Pu Zhang, Cheng Yang, Zhuosheng Zhang
Computation and Language · Cross-listed from cs.AI
Metaphor Tracer: A Theory-Informed Analysis of Hidden States
Marc Heimann, Roxana Assadi Moghaddam, Olga Brovkina et al. (+2 more)
What do a
language model's hidden states say about the organization of a single text? From one forward pass, without training, we score every
token position on two properties. The *aggregator* measures whether the position consolidates the whole text into a stable configuration. The *differentiator*, whether other
tokens are transiently carried into its subspace as the model reads: metaphor in its root sense, transport. Constants were frozen on one discovery text; every other is confirmatory. The aggregator is not, in the classic sense, an information measure, nor a measure of salience. Across three unrelated models, as a signifier repeats, its surprisal and its
attention drain while its aggregator score holds: the channel marks a
token's place in the text. That this tracks a reading rests on independent ground truth: an engineered register the aggregator follows across its boundaries (6/6
cells), and a psychoanalyst's marking of clinical transcripts, fixed before the instrument existed, in 34/36
cells, with a graded increment above lexical controls and dissociations no type-level measure reproduces. A transfer test gives the result its shape: the model whose
token structure travels with lexical type reads the singular discourse worst, and in a matched base/instruct pair tuning raises fidelity without moving type-transfer. Structural value is a property of a
token's place in *this* text, not of its vector alone: a relational rather than essentialist reading of hidden states, operationalizing theory that predated the instrument.
Read the full abstract ▾
All authors Marc Heimann, Roxana Assadi Moghaddam, Olga Brovkina, Mark Pettifor, Lutz Goetzmann
Computation and Language · Cross-listed from cs.AI
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Haozhe Hu, Hao Wu, Peiran Yin et al. (+3 more)
Pruning is a promising approach for improving the efficiency of large
language models (LLMs). Static structured
pruning is hardware-friendly, but its input-agnostic computation allocation often degrades model quality at high sparsity. Dynamic
pruning adapts computation to individual
tokens, but coarse-grained layer-level routing limits allocation flexibility, and finer-grained decisions introduce irregular execution that can offset computational savings. To address these challenges, we present WIDE, the first end-to-end differentiable framework that combines
token-wise structured dynamic width
pruning with GPU kernel co-design for both prefill and decoding. WIDE uses differentiable routers to select GQA-aligned query-head groups and GPU-tile-aligned FFN-channel groups for each
token. To execute these decisions efficiently, it reorders routing masks to cluster active
tokens within each group, enabling hardware-agnostic block-level skipping and architecture-aware skipping of inactive loads and tensor-core operations. Experiments on the Llama3 and Qwen3 families demonstrate improved quality retention over representative static and dynamic
pruning methods. At 50% target routing sparsity, WIDE improves Llama3.1-8B average zero-shot accuracy by up to 20.26 points over the strongest dynamic-depth baseline under calibration-only settings, while retaining 97.6%/88.2% of dense Qwen3-14B likelihood/generation performance after LoRA tuning. Separate latency
benchmarks demonstrate end-to-end speedups of up to $1.68\times$ for prefill and $1.55\times$ for decoding over the dense baseline. These results demonstrate that
pruning–kernel co-design pushes the frontier of
token-wise dynamic structured
pruning, combining improved quality retention with practical acceleration across both prefill and decoding. Our code is avaliable at \url{https://github.com/EIT-NLP/LLM-
Pruning/tree/main/WIDE}.
Read the full abstract ▾
All authors Haozhe Hu, Hao Wu, Peiran Yin, Chao Han, Yunpu Ma, Xiaoyu Shen
Computation and Language · Cross-listed from cs.CE
Can Large Language Models Execute Parent Orders?
Zane Shen, Xinli Xu, Guangyi Zhang et al. (+7 more)
Parent-order execution is a core problem in algorithmic trading, where the goal is to split a large order into smaller orders while reducing execution costs. Existing approaches either rely on pre-specified
market assumptions that may not hold in practice, or require task-specific training that limits adaptability to new settings. To overcome these limitations, we present the first systematic study of large
language models (LLMs) for parent-order execution. This extends the use of LLMs in finance from what to trade to how to execute. We propose PACE (Plan-Ahead Controlled Execution), a hierarchical framework that decomposes parent-order execution into long-horizon planning and short-horizon execution, requiring neither explicit
market assumptions nor task-specific training. Experiments on Shenzhen Stock Exchange Level-1 data show that PACE outperforms TWAP, Almgren-Chriss, and learning-based baselines, exceeding the strongest baseline by 0.65 bps. Behavioral analysis reveals that LLMs make execution decisions differently from human investors: higher model confidence predicts better performance rather than worse returns, and the model trades earlier rather than procrastinating toward the deadline. These findings suggest that LLMs can complement human traders in execution decisions.
Read the full abstract ▾
All authors Zane Shen, Xinli Xu, Guangyi Zhang, Jialong Chen, Jinsong Zhou, Cong Chen, Guibao Shen, Dongyu Yan, Luozhou Wang, Zhen Yang
Computation and Language · Cross-listed from cs.AI
GLM-RAG: Graph Language Models for Graph-Based Retrieval-Augmented Generation
Maya Arseven, Anette Frank, Beni Egressy et al. (+2 more)
Retrieval-augmented generation (RAG) over knowledge graphs requires retrievers that can effectively capture both graph structure and semantic information. Recent approaches have explored graph neural network (GNN)-based retrievers to model graph
topology in multi-hop
reasoning tasks. In parallel, graph
language models (GLMs) have emerged as a promising paradigm that integrates graph
reasoning and the semantic capabilities of
language models. In this work, we introduce a GLM-based retriever and investigate the comparative strengths of GLM-based, GNN-based, and traditional vector-search-based retrievers in single- and multi-hop RAG settings, and with a particular focus on transferability to unseen domains. Our findings suggest that finetuned GLM retrievers generalize better out of domain, achieving SOTA on two multi-hop
benchmarks. On in-domain multi-hop QA
datasets they remain comparable to prior work, with promising scaling as parameters and subgraph coverage increase. GNN-based retrievers achieve higher graph coverage with an efficient training setup, whereas the vector-search baseline excels at single-hop
datasets.
Read the full abstract ▾
All authors Maya Arseven, Anette Frank, Beni Egressy, Johann Higl, Moritz Plenz
Computation and Language
Correlation between prosody and pragmatics: A case study of the discourse marker hālā `now' in Persian
Soleiman Ghaderi, Moloud Asakereh and Kevin Tang
The Persian discourse marker hālā ('now') exhibits remarkable multifunctionality, extending far beyond its temporal adverbial role to encompass a variety of pragmatic functions. This study presents a pragmatic and acoustic analysis of hālā in spoken Persian, examining 267 instances from spontaneous conversations. While temporal uses were present, they were often combined with other discourse marker functions, indicating extensive multifunctionality, with 70% of
tokens serving two or more pragmatic roles. Textual functions (topic shifting, signaling relationships, boundary marking,
attention guidance, topic introduction, and topic emphasis) were most frequent, followed by interactive functions (turn management, listener engagement, and feedback regulation), and modal functions (epistemic stance, emotional expression, and attitudinal marking). Prosodic analysis revealed that duration and intensity are key cues for distinguishing hālā's functions. Textual uses were significantly shorter, while temporal uses showed a tendency toward longer realizations. Interactive functions correlated with higher intensity, while modal functions showed a weaker tendency toward lower intensity. These findings indicate that duration and intensity are the main prosodic cues associated with functional differentiation in hālā, especially in textual and interactive uses.
Read the full abstract ▾
Computation and Language
LLMs struggle to simulate human belief updates in controlled environments
Sebastian Pohl, Harsh Mehta, Pranav Mambayil et al. (+4 more)
LLMs are increasingly deployed as proxies for human study participants in social science experiments, yet the fidelity of this practice has rarely been tested directly. We test whether six LLMs can simulate individual human belief updates, comparing LLM outputs 1-to-1 against ground truth data from 391 UK participants on Prolific, who updated their stances on three discussion topics after reading Reddit comments. Each participant was simulated by an LLM conditioned on a persona derived from their demographic and personality trait data. We find that some LLMs (Qwen3-32B and GPT-5-Mini) can match the human post-stance distribution, but only when given participants' actual initial stances. All six models fail to simulate initial stances themselves and to produce faithful belief updates from self-generated stances. Three systematic biases emerge across all models: overrepresentation of neutral positions, more frequent but smaller belief shifts than humans, and a failure to rank comments by convincingness. Demographic and personality trait personas had no consistent effect on fidelity. LLM simulations of human belief dynamics are only reliable when grounded in realistic starting conditions, that current multi-round social media simulations rarely provide.
Read the full abstract ▾
All authors Sebastian Pohl, Harsh Mehta, Pranav Mambayil, Abdul Ghafoor, Franziska Lesigang, Yufang Hou, Christian Hilbe
Computation and Language
Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations
Pere Martra, Eugenio Martínez Cámara and Alfonso Ureña López
This work presents
Fairness Pruning, a lightweight structural intervention method designed for the management and future mitigation of demographic bias in large
language models (LLMs). As a foundational empirical validation of this method, this work focuses on causal bias localization. Using minimally contrastive
prompt pairs and inference-time activation capture, the method identifies
neurons that react differentially when processing demographic attributes in GLU architectures, evaluating the signal at the down_proj input. Empirical evaluation was conducted on models of up to 3 billion parameters (Llama-3.2 family and Salamandra-2B), combining standardized
benchmark evaluation with qualitative text generation experiments. Results demonstrate that zeroing the identified
neurons alters how the model responds to associated demographic variables. However, rather than producing flat mitigation, the intervention causes bidirectional bias destabilization: because BiasScore is unsigned, candidate sets mix
neurons that push toward and against the stereotype, and the net effect on aggregate bias depends on which sign dominates. The intervention is extremely
surgical: zeroing at most 40
neurons in Llama-3.2-1B (less than 0.031% of total MLP width) achieves a mean retention of 99.49% in
reasoning and general knowledge capabilities. These findings empirically confirm that demographic bias processing and model capabilities operate on dissociable
circuits, establishing the methodological foundations for transitioning from blind zeroing toward directional behavior modulation.
Read the full abstract ▾
Computation and Language
CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance
Anubhav Lakra and Yue Feng
Large
Language Models (LLMs) deployed in dynamic financial environments face a critical challenge: maintaining factual accuracy as
market conditions, regulations, and corporate facts change continuously. While 4-bit quantization enables efficient deployment, it severely limits the viability of sequential
memory editing: existing methods undergo catastrophic performance degradation under this "quantization stability crisis." We introduce CACHE-UK (Contextual Adaptive Continual Hybrid Editor for UK Finance), a stability-aware
memory editing framework specifically designed for domain-specific, quantized LLMs. CACHE-UK integrates three components: a rank-1 LoRA perturbation mechanism that confines edits to the low-rank adapter subspace, a financial domain prioritization module for content-adaptive edit strength, and a closed-loop Stability Controller that tracks "degradation debt" to prevent catastrophic forgetting across sequential updates. Evaluated on a 4-bit quantized OpenLLaMA-3B model with a curated UK financial corpus of 88,021 documents, CACHE-UK reduces knowledge degradation by 11-17% relative to adapted baselines under identical 4-bit constraints – its most robust effect – while attaining the highest test success (generalization) rate observed in our setting (28%, a 6 percentage point improvement over the strongest adapted baseline). These results indicate that stability-aware editing can improve factual maintenance in resource-constrained financial LLM deployments, though absolute generalization rates remain low.
Read the full abstract ▾
Computation and Language
(Towards) Scalable Reliable Automated Evaluation with Large Language Models
Bertil Braun and Martin Forell
Evaluating the quality and relevance of textual outputs from Large
Language Models (LLMs) remains challenging and resource-intensive. Existing automated metrics often fail to capture the complexity and variability inherent in LLM-generated outputs. Moreover, these metrics typically rely on explicit reference standards, limiting their use mostly to domains with objective
benchmarks. This work introduces a novel evaluation framework designed to approximate expert-level assessments of LLM-generated content. The proposed method employs pairwise comparisons of outputs by multiple LLMs, reducing biases from individual models. An Elo rating system is used to generate stable and interpretable rankings. Adjustable agreement thresholds, from full unanimity to majority voting, allow flexible control over evaluation confidence and coverage. The method's effectiveness is demonstrated through evaluating competency profiles extracted from scientific abstracts. Preliminary results show that automatically derived rankings correlate well with expert judgments, significantly reducing the need for extensive human intervention. By offering a scalable, consistent, and domain-agnostic evaluation layer, the framework supports more efficient and reliable quality assessments of LLM outputs across diverse applications.
Read the full abstract ▾
Computation and Language
MORFES: A Benchmark for Productive Inflectional Competence in Modern Greek
Ioakeim Perros, Cleopatra Papadopoulou, Ayoub Kirouane et al. (+1 more)
Modern Greek is a richly inflected
language, yet the
language models built for it are evaluated mainly on factual knowledge, and no
benchmark is dedicated to their inflectional competence. We introduce MORFES (Morphological Open-class Recognition-and-Formation Evaluation Suite), a
benchmark of 500 expert-verified items that tests the recognition and production of Greek inflected forms, favoring lower-frequency lemmas so that a correct answer reflects the rule rather than a memorized form. We make it publicly available at https://huggingface.co/
datasets/KIEFERSA/MORFES. We evaluate a range of open
language models on MORFES, situating them within the rapidly scaling open-weight ecosystem from LLaMA to Qwen3, DeepSeek-R1, Magistral, and Kimi K2, where
multilingual coverage grows but grammatical competence in morphologically rich
languages remains under-measured. Among them, Sophea-Genesis-1, a model we developed and release as open weights at https://huggingface.co/KIEFERSA/Sophea-Genesis-1, leads on inflectional morphology while matching similarly sized models in general capability.
Read the full abstract ▾
All authors Ioakeim Perros, Cleopatra Papadopoulou, Ayoub Kirouane, Christos Petrocheilos
Computation and Language
CoMem: Reusing Transformer Depth across Queries with Persistent Intermediate Residuals
Hanzuo Liu, Xuan Qi, Chunyu Liu et al. (+6 more)
Repeated queries over shared documents repeatedly execute the same lower
transformer layers. We introduce CoMem, which makes split depth j an explicit reusable-context axis: write one depth-j residual per
token, select a bounded chunk set, and resume only layers [j:L). Among document-reuse systems we are aware of, CoMem jointly makes split depth a tunable serving axis and isolates it with a matched j=0 endpoint. On Qwen3-8B, j=12 reduces selected-pack Read from 931.9 to 664.4 ms (1.403x), with a 3.12-point RULER cost (95% CI [2.36, 3.93]); a continuous-prefix oracle recovers the full gap. The resulting depth axis quantifies a quality-latency-storage trade-off; a separate same-adapter, Write-inclusive pipeline is 2.74x faster. Equal-latency raw replay leads by 11.56 points with BM25, directly measuring an applicability boundary of prepaid depth rather than hiding it. CoMem stores 8 KiB/
token versus 144 KiB/
token for a protocol-aligned same-Qwen3 CacheBlend-style diagnostic; the cohorts and adaptation budgets are not matched. A context-position factorization identifies missing lower-layer document context as the dominant tested multikey error, and a 32-
token overlap raises 92.5 to 98.5 without increasing persistent bytes or per-query Read. CoMem opens
transformer depth as a measurable, tunable dimension for repeated-query long-context serving.
Read the full abstract ▾
All authors Hanzuo Liu, Xuan Qi, Chunyu Liu, Haotian Zhong, Yulong Wang, Key, Rayying, Alex Lamb, Mingyu Gao
Computation and Language · Cross-listed from cs.AI
CDAE: Enhancing Perturbation Robustness in Pretrained Language Models with Contrastive Denoising
Sina Heydari, Amirreza Abbasi, Mohsen Hooshmand et al. (+1 more)
Pre-trained
language models have significantly improved sentence representation learning, yet their
embeddings remain sensitive to textual perturbations such as synonym substitution, masking, and word dropout. This work proposes a lightweight Contrastive Denoising Autoencoder (CDAE) that refines pre-trained BERT
embeddings by jointly optimizing contrastive and reconstruction objectives to learn perturbation-invariant representations. We evaluate the proposed framework under multiple perturbation strategies and strengths, and compare it against the original BERT
embeddings and SimCSE. Experimental results show that CDAE consistently preserves higher
embedding similarity under perturbation, with the gap widening as the perturbation strength increases, highlighting perturbation-invariant learning as a promising direction for improving the stability of sentence
embeddings. The source code is publicly available at: https://github.com/ComputationIASBS/CDAE
Read the full abstract ▾
All authors Sina Heydari, Amirreza Abbasi, Mohsen Hooshmand, Majid Ramezani
Computation and Language
EMBL AI Librarian: Life-Sciences Knowledge Layer for AI Agents
Luigi Sigillo, Matteo Silvestri, Francesco Tabaro et al. (+9 more)
The web is increasingly accessed by AI
agents rather than humans. Every
agent needs knowledge, especially in the life-sciences, where agentic pipelines are growing fast. Access to the literature is a crucial part of that need, and resources such as Europe PMC, with over 40M indexed records, are widely used to meet it. Yet these resources were not built for AI
agents: they take keywords and complex syntax and return whole papers, so every
agent must learn the syntax, issue several searches, and read full papers to find the evidence it needs. We introduce EMBL AI Librarian, a knowledge layer that upgrades the Europe PMC interface for AI
agents: an
agent asks in natural
language and receives evidence that answers it. A single LLM orchestrates the whole knowledge
retrieval process: it plans complementary subqueries executed by the live Europe PMC search engine, then reads the selected papers and locates the relevant evidence. We evaluate Librarian across four
benchmarks: literature synthesis, claim verification, open-domain question answering, and downstream biology tasks such as protocol questions and sequence manipulation. On ScholarQABench, Librarian improves Citation F1 by more than $16$ points over strong recently published baselines. Used as the
retrieval layer of an existing claim-verification pipeline, it increases agreement with expert consensus; and on the open-form LitQA2
benchmark, a GPT-5.4
agent scores about $8$ points higher when grounded in Librarian than with web search. Overall, our results show that equipping life-science
agents with the Librarian knowledge layer improves performance across a range of tasks. We release our code publicly at https://github.com/petroni-lab/librarian
Read the full abstract ▾
All authors Luigi Sigillo, Matteo Silvestri, Francesco Tabaro, Rajat Bhatnagar, Syed Irtaza Mubashar, Matt Jeffryes, Daljit Nijjer, Vittorio Perera, Ola Spjuth, Julio Saez-Rodriguez, Melissa Harrison, Fabio Petroni
Computation and Language
Causal Discovery with Inverted Self-attention for Multivariate Time Series
Yusen Liu, Yong Wang, Yifan Yin et al. (+3 more)
Causal discovery in multivariate time series data is challenging due to complex interactions, high dimensionality, and nonlinear dependencies among variables. Existing methods often struggle to capture these complexities, resulting in inaccurate causal structures. To address this issue, we propose a novel framework that leverages self-
attention mechanisms within the
transformer architecture for causal discovery. Our approach introduces a novel inverted causal self-
attention mechanism (CSAM) that emphasizes latent and indirect causal relationships by inverting
tokens and inducing sparsity in
attention scores, focusing on significant causal interactions and reducing spurious correlations. Additionally, we develop a global causal
algorithm to identify global causal links, providing a holistic metric for causal influence, along with a causal verification module to ensure robustness in the identified causal relationships, enhancing the reliability of our framework. Experiments on both linear and nonlinear
datasets, along with ablation studies and sensitivity analyses, show that our framework outperforms existing methods, demonstrating its potential for causal discovery in complex multivariate time series.
Read the full abstract ▾
All authors Yusen Liu, Yong Wang, Yifan Yin, Tianqing Zhu, Xiufeng Liu, Huan Huo
Computation and Language
Fidelity Is Not Safety: Gently-Compressed LLMs Pass Every Data-Free Quality Guard Yet Invent Procedure Steps in Agentic Execution
I. Kennedy and T.Kennedy
Practitioners accept a compressed
language model once it clears a stack of data-cheap quality guards: perplexity within a small factor of the original, downstream accuracy (for example MMLU) inside a confidence interval, and data-free output-fidelity signals that compare the compressed and original network's internal representations under random probe inputs. This stack has a blind spot. Across three model families, gently-compressed models clear every guard and then invent procedure steps that were never in the instructions when they run a standard operating procedure (SOP) as an
agent. The effect is operator-specific: coherent low-rank (SVD) truncation induces it, and magnitude
pruning matched to the same perplexity does not. One dissociation isolates the cause. The same compressed weights that CI-win a paired output-fidelity test CI-fail the invented-step canary. The governing axis is the coherence of the compression error times its rate; the magnitude of the damage does not predict it. The data-free fidelity probe is a fidelity oracle by construction, so it cannot see this axis. We characterize the blindspot and dissociation with paired confidence intervals on a pre-registered, powered canary across three architectures. Operator-specificity replicates on all three, and the perplexity-guard evasion appears where the model admits in-guard low-rank headroom. We then give a data-free screen: a two-axis statistic of the compression error (coherent-fraction and error-rate) that flags the failing builds with fixed thresholds across architectures and matches the coherence-times-rate mechanism. Perplexity, MMLU, and fidelity acceptance do not certify
agent safety. Screen gently-compressed low-rank builds before agentic deployment
Read the full abstract ▾
Computation and Language
The MADRS Pipeline: Supporting Depression Assessment in Clinical Trials
Mila Fodor, Katalin Ócsai, Francesco Periti et al. (+2 more)
Depression is a major mental disorder for which diagnosis relies primarily on clinical assessments. Automated methods to support its detection via the psychiatric MADRS scale are getting more and more
attention. While existing solutions primarily focus on detecting the disorder from different text sources (e.g., online text, social media), there is still limited support for clinical trials, where clinical assessments are conducted through structured interviews based on standard guidelines such as SIGMA. In this work, we develop a LLM pipeline specifically designed to support clinicians in supporting the assessment of depression in patients enrolled in clinical trials. Our pipeline converts audio interviews into transcripts, maps them into the ten MADRS symptom items, estimates their severity, and identify problematic clinical ratings associated with them. Evaluation on real clinical interviews shows a strong overall correlation of 0.867 with expert ratings, providing interpretable support for future assessments in clinical trials.
Read the full abstract ▾
All authors Mila Fodor, Katalin Ócsai, Francesco Periti, Rien Sonck, Alex Boudreau
Computation and Language
Commit Locally, Exit Globally: Coordinating Adaptive Sampling and Early Exit in Diffusion Language Models
Chia-Ming Lee, Shao-Kai Liu, Ming-Ching Chang et al. (+3 more)
Diffusion language models predict
tokens in parallel but still require many iterative forward passes. Adaptive sampling accelerates local
token commitment, while recent global stopping methods seek to end generation early. Our trajectory analysis shows that short answers can stabilize early in decoding, whereas
reasoning outputs often require more steps to settle. A fixed progress threshold therefore struggles to avoid both redundant computation on short-answer tasks and premature termination on
reasoning tasks. We introduce C$^4$, a training-free framework that combines local decoding acceleration with adaptive global early exit. Confidence-Verified Early Exit (CVEE) determines when to end generation, skipping the remaining steps once its stopping criterion is met. It combines confidence and temporal agreement to adapt stopping to the observed revision history. Commit-Core-Then-Confirm (CCTC) accelerates
token commitment while generation continues, reducing the decoding steps needed to complete each block. It extends confidence-threshold parallel decoding with a boundary-anchored core and one-step confirmation of deferred predictions, constraining additional commitments to mitigate premature freezing under unresolved context. With CVEE calibrated once on LLaDA, the same numerical settings transfer across all 12 zero-shot tasks and to Dream without recalibration. Under this configuration, C$^4$ reduces decoding steps by 64-95% and achieves measured decoding-loop speedups of 2.6x-18.6x, with task performance close to full decoding. Component analyses show complementary contributions: early exit supplies most savings on short-answer tasks, while local acceleration contributes more on
reasoning and code generation.
Read the full abstract ▾
All authors Chia-Ming Lee, Shao-Kai Liu, Ming-Ching Chang, Xin Li, Yu-Lun Liu, Chih-Chung Hsu
Computation and Language
Jingxiang Fan, Junbao Zhuo and Bochao Zou
Existing
multimodal long-term
memory agents use external
memory to overcome the limited context available for long
videos. However, most methods emphasize what to store rather than how stored
memory should be retrieved. When
retrieval becomes inaccurate or repeatedly fails to obtain useful evidence, existing
agents lack mechanisms to diagnose failures from previous task trajectories and adapt future search strategies.We introduce Reflective
Retrieval Memory (RRM), a reflective
memory framework for long-horizon
multimodal reasoning. RRM augments an entity-centric
multimodal memory graph with reflective experience
memory, which distills transferable procedural
retrieval knowledge from historical task trajectories. Unlike episodic and semantic
memories that preserve factual evidence from the current
video, reflective experience
memory captures reusable search strategies across tasks. RRM converts retrieved experiences into query-level guidance, while answer generation remains conditioned only on factual evidence newly retrieved from the current
video. A lifecycle management mechanism further regulates experience
memory through usage frequency, reuse feedback, and temporal decay, thereby reducing redundancy and noise. RRM consistently outperforms previous state-of-the-art approaches on M3-Bench-
Robot, M3-Bench-Web, and
Video-MME-Long, demonstrating the effectiveness of reflective
retrieval memory for long-horizon
multimodal reasoning.
Read the full abstract ▾
Computation and Language
Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game
Niklas Bauer, Lars Benedikt Kaesberg, Akiko Aizawa et al. (+3 more)
As large
language models (LLMs) are deployed as
agents in high-stakes settings, such as medical and legal systems, understanding their deceptive capabilities is fundamental to safety. Controlled social deduction games provide a reproducible proxy for isolating and evaluating these complex
adversarial behaviors. We present the open-source
benchmark framework ParliamentBench based on the game Secret Hitler to evaluate LLMs in scenarios that require deception, persuasion, and
reasoning under information asymmetry. We evaluate 16 LLMs across 1,600 simulated matches playing each other, playing against humans, and compare them against a large set of online games. We introduce three novel metrics that isolate social deduction,
reasoning, and deceptive consistency. Our experiments reveal that frontier models achieve strong performance across cooperative and deceptive roles, with a strong top-four cluster (GPT-5.4, Kimi K2.5, Grok 4.1 Fast, and DeepSeek 3.1 Terminus), whereas the weakest models fall short of random (33%) and simple algorithmic (45%) baselines. Most LLMs struggle to maintain a consistent deceptive persona throughout an entire game, with deception retention dropping below 50%.
Read the full abstract ▾
All authors Niklas Bauer, Lars Benedikt Kaesberg, Akiko Aizawa, Jan Philip Wahle, Bela Gipp, Terry Ruas
Computation and Language
Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models
Shuyi Fan, Boyuan Deng, Mengyu Xu et al. (+4 more)
LLM tutoring poses a measurement problem: can a general-purpose helpfulness rubric distinguish direct answer-giving from pedagogical guidance? We audit this signal in a pre-registered study. Within each of three tutor bases, we compare conversational and pedagogical policies instantiated with the same underlying model and paired with one fixed weak simulated student. Deterministic detectors measure answer leakage and next-turn independent work. Claude Opus 4.8 is the frozen, condition-blind primary judge. After the Opus scores were fixed, GPT-5.6 Sol was prospectively specified for a post hoc robustness audit of the same 1,179 confirmatory answer-phase tutor turns under the frozen helpfulness and pedagogy rubrics. On the primary base under Opus, the policies do not differ significantly in helpfulness but are perfectly rank-separated under the pedagogy rubric (Cliff's $|\delta|{=}0.10$ vs. $1.0$). Across the two judges, pedagogy contrasts retain their direction where detected, whereas the helpfulness ordering is judge-contingent, reversing between judges on two of three bases. In an Opus-only ablation, seven primary-base policies span $2.3$ points in mean judged pedagogy within a $0.25$-point band of mean judged helpfulness. Separately, answer-revealing turns are followed by less independent student work on every base, a result that is judge-invariant by construction. In this controlled setting, general-purpose helpfulness is not a reliable pedagogy signal. Tutor evaluation should pair pedagogy-targeted rubrics with deterministic process measures.
Read the full abstract ▾
All authors Shuyi Fan, Boyuan Deng, Mengyu Xu, Jiale Liu, Hongyang Zhang, Qiaoxin Yang, Chongyang Gao
Computation and Language
FinSMART: Financial Sentiment Analysis for Algorithmic Trading through Market-Aligned Reinforcement Learning
Giorgos Iacovides, Wuyang Zhou and Danilo Mandic
Recent advances in Generative AI have substantially improved financial sentiment analysis through post-trained financial large
language models (LLMs). However, existing approaches remain confined to a
market-agnostic, supervised learning paradigm that relies on limited, static and human-annotated
datasets, and thus are incapable of adapting to evolving
market conditions. To address this limitation, we introduce FinSMART, the first
market-aligned reinforcement learning framework for financial sentiment analysis, which directly optimizes sentiment signals using realized
market outcomes. To deal with the noisy, non-stationary, and multifactorial nature of financial
markets, FinSMART incorporates a signal extraction pipeline that combines
market-aware data filtering with a discrete asymmetric trading reward, enabling stable reinforcement learning from economically meaningful
market feedback. Experimental results demonstrate that FinSMART significantly outperforms existing state-of-the-art methods in profitability, risk-adjusted performance, and sentiment signal quality, improving cumulative trading returns by 220% over the strongest baseline. Uniquely, the FinSMART framework naturally supports
market-aware retraining, at any point in time, by replacing costly manual annotation with newly observed financial articles and their realized
market outcomes. Such a retraining strategy enables the model to continuously adapt to changing
market dynamics, resulting in consistent performance gains over its static counterpart. These findings demonstrate the practical applicability of
market-aligned reinforcement learning and highlight its potential as a next-generation paradigm for developing adaptive financial LLMs.
Read the full abstract ▾
Computation and Language
Challenges in annotations by humans and LLMs: A case study of evaluative language
Mirela Imamovic, Aenne Cecilia Kristine Knierim, Khushi Pitroda et al. (+1 more)
In this paper, we draw a comparison between linguists in training, a trained linguist, and annotations generated by large
language models (LLMs) to find out if they struggle with complex linguistic phenomena in a similar way. For this purpose, we analyse evaluative
language in spoken popular science discourse, with the example of a corpus of English TED talk transcripts. We focus on the Appraisal theory and its Attitude subsystem, including the categories (classes) of Affect, Judgement, and Appreciation. In this context, Appraisal theory is an example of a highly subjective annotation task, making it a suitable example for the study of complex annotation challenges. First, we assess human annotations on a sentence level in specific scientific domains. Then, we develop three
prompts and compare them for model performance for the automatic classification of Appraisal classes. We assess the performance of three LLMs using the best-performing
prompt and finetune the model, reaching an F1-score of 0.77. We find that models perform best compared to annotations conducted by the trained linguist, while linguists in training do not reach high agreement scores. We conclude that LLMs can aid in complex annotation task resolution, opening new pathways for the complex theories annotated and analyzed in digital humanities studies.
Read the full abstract ▾
All authors Mirela Imamovic, Aenne Cecilia Kristine Knierim, Khushi Pitroda, Ekaterina Lapshinova-Koltunski
Computation and Language · Cross-listed from cs.NI
PCAP-LM: An LLM-Native Text Representation for TLS Bulk Traffic Analysis
Xavier Marjou, Lucas Tamic and Ilan Jaffeux-Cheniout
Large
language models (LLMs) offer powerful
reasoning capabilities for network traffic analysis, but standard capture formats and their textual equivalents are prohibitively verbose, overflowing LLM context windows by two orders of magnitude. We present PCAP-LM, a flow-centric, LLM-native text representation that acts as a lossy knowledge extraction step rather than a standard compression tool: raw captures are transcoded into semantic summaries using PacketGlyphs - a novel ASCII alphabet coined in this paper that encodes packet direction, TCP/TLS state, log-scale size, and inter-packet delay. Combined with a constrained PMI-BPE tokenizer and motif run-length encoding, repetitive behavioural patterns are aggressively collapsed. A @REFS side-index preserves lossless drill-down into the original packets. Evaluated on a homogeneous corpus of 5G/4G TLS 1.3 bulk-download traffic, the BPE vocabulary fully saturates at 159
tokens, achieving an 812x size reduction over tshark -V and fitting entire captures within a single LLM context window. In a forensic question-answering evaluation over 30 held-out files, a frontier LLM achieves 99.3% accuracy from PCAP-LM documents versus 51.0% from a
token-budget-matched tshark -V prefix. The lossy design introduces known blind spots - most notably a 24% false-negative rate for TCP retransmissions - and extending to heterogeneous mixed-protocol environments will require vocabulary retraining.
Read the full abstract ▾
Computation and Language
GGC: Selective Query Correction for Reliable Text-to-SPARQL Generation
Ziyi Yang, Thanh-Son Nguyen, Tuan Anh Nguyen et al. (+1 more)
Large
language models (LLMs) have demonstrated strong capabilities in structured query generation, making them a natural choice for Text-to-SPARQL, which translates natural
language questions into executable SPARQL queries over knowledge graphs. However, their initial outputs remain unreliable: generated queries may be executable yet semantically misaligned with input questions, leading to incorrect
retrieval. To address this issue, we propose Generator-Gate-Corrector (GGC), a framework for reliable LLM-based Text-to-SPARQL generation. GGC first uses a Generator to produce an initial query, then applies a Gate to predict whether correction is needed, and finally invokes a Corrector only for selected high-risk queries. This selective correction mechanism avoids unnecessary modifications and reduces the risk of degrading originally correct queries. Experiments on MCQA show that GGC improves query-level accuracy from 90.23% to 98.33% while reducing inference overhead by 45% compared with correcting all generated queries. Ablation studies show that the Gate is robust across thresholds and that Corrector training data composition affects correction effectiveness and stability. Overall, the results demonstrate that selective correction enhances the accuracy, reliability, and efficiency of LLM-based text-to-SPARQL generation.
Read the full abstract ▾
All authors Ziyi Yang, Thanh-Son Nguyen, Tuan Anh Nguyen, Lihui Chen
Computation and Language
LEEPS: Latent-Guided Explore-Exploit Prompt Sampling for Efficient RLVR in Large Language Models
Shuang Liang, Haoyang Zhou, Yifan Gong et al. (+2 more)
Reinforcement learning with verifiable rewards (RLVR) improves the
reasoning capabilities of large
language models, but
prompt groups with identical rollout rewards consume generation budget without effective learning signals. Pre-rollout
prompt selection can reduce this waste by screening
prompts before rollout generation. However, existing pre-rollout methods struggle to balance exploitation and exploration: repeatedly exploiting historically informative
prompts can narrow training coverage, whereas broader exploration can lower the fraction of informative
prompts. To address these limitations, we introduce LEEPS, a Latent-Guided Explore–Exploit
Prompt Sampler that adaptively balances the reuse of previously observed informative
prompts with continued exploration of uncertain ones. LEEPS partitions candidates into exploit and explore portfolios and adaptively allocates rollout budget according to their recent non-trivial ratios. It further uses representation-space neighbors and historical rollout outcomes to prioritize uncertain
prompts likely to yield non-zero reward variance, thereby making exploration more targeted without additional rollouts. Across six mathematical
reasoning benchmarks, LEEPS achieves the highest average score at both model scales, with relative gains of 2.6% and 3.7% over the strongest baseline for Qwen2.5-Math-1.5B and 7B, respectively, and generally improves faster during the training process. It also achieves the highest average score across the three evaluated OOD general-
reasoning benchmarks at both model scales and adds only about 2 seconds of online sampling overhead per training step. Code is available at https://github.com/ShuangLiangX/LEEPS.
Read the full abstract ▾
All authors Shuang Liang, Haoyang Zhou, Yifan Gong, Guowei Wang, Xiting Wang
Computation and Language
RepBench: Compiling Benchmarks into Capability Representations for Large Language Models
Yanshi Li, Xueru Bai, Shuman Liu et al. (+1 more)
Representation engineering reads and steers capability directions in large
language models, yet methods are typically evaluated on paper-specific synthetic data. The resulting measurements are difficult to compare or reproduce and may reflect surface patterns rather than capabilities. We present RepBench, a
benchmark-grounded data layer for capability-aligned representation probing. Crawling 13,427
benchmark papers yields a taxonomy of 182 capability clusters in 13 families; harvesting 353 public
benchmark datasets yields 46,149 audited probe texts covering 94 capabilities, each supported by at least two independent
benchmarks. This multi-
benchmark design reduces dependence on any single source: raw per-text vectors exhibit no natural cluster granularity, whereas
benchmark-pooled capability vectors show an interior clustering optimum at a small number of clusters on all 12 evaluated models, with low agreement to the human taxonomy. Under cross-
benchmark transfer evaluation across twelve models completed by all four readouts, difference-in-means attains the highest model-level mean on ten models, while logistic regression wins the most capability-model
cells. This disagreement shows that the readout method and aggregation criterion are meaningful evaluation dimensions. The pipeline, corpus, and evaluation code are released as a reusable closed-loop workflow.
Read the full abstract ▾
All authors Yanshi Li, Xueru Bai, Shuman Liu, Long Zhang
Computation and Language · Cross-listed from cs.DL
SciSchema.org: A Multidisciplinary Collection of Schemas for Structured Scientific Process Descriptions
Jennifer D'Souza, Sameer Sadruddin, Anisa Rula et al. (+23 more)
Scientific processes are often described in heterogeneous article discourse, with details needed for comparison, reproducibility, reuse, and automation dispersed across prose, tables, figures, protocols, and supplementary files. We present the first release of SciSchema.org, a multidisciplinary collection of 16 expert-annotated schemas spanning Biology & Biotechnology, Materials & Chemistry, Imaging & Measurement, Physics, and Psychology. Each schema defines reusable fields for describing process instances, including inputs, outputs, materials, instruments or software, parameters, conditions, procedural steps, measurements, and provenance-related information. The schemas were created through a human-in-the-loop schema-mining workflow in which large
language models generated candidate structures from process specifications, scientific articles, and expert feedback, followed by domain-expert construction of final master schemas. The
dataset contains final schemas in JSON Schema and SHACL formats, intermediate model-generated schemas, expert-feedback records, source-paper metadata, community-development materials, and analysis scripts. Technical validation assessed schema structure, development provenance, expert review, and syntactic conformance. The collection supports structured annotation, metadata enrichment, scientific knowledge graphs, information extraction, semantic publishing, and cross-study comparison.
Read the full abstract ▾
All authors Jennifer D'Souza, Sameer Sadruddin, Anisa Rula, Ana Bossler, Andrés Fullana, Enric Bas, Syed Ather, Defne Circi, Anlan Chen, L. Catherine Brinson, Alyssa Columbus, George Demetriou, Dongjun Jeong, Tarun Kumar, Frank Krüger, Sascha Genehr, Kai Budde-Sagert, Anamaria Leonescu, Francesco Lodola, Chiara Florindi, Gagana Balasubramanya Murthy, Samson Oluwapelumi Olagbile, Nazia Riasat, Yan Sha, Kevin Shen, Shaokai Yang
Computation and Language · Cross-listed from cs.LG
TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement
Cheng Wei
Federated fine-tuning of large
language models (LLMs) enables collaborative training without exposing raw data. However, a recent attack, NeuroImprint, demonstrates that a malicious parameter server can corrupt a PEFT adapter into a
privacy backdoor: by assigning a dedicated memorization
neuron to each training sample and ensuring each
neuron updates at most once, the server can analytically reconstruct 59%–79% of client training data with high semantic fidelity. Existing defenses—including local differential
privacy (LDP) and gradient clipping—either fail against this attack or impose unacceptable utility degradation. We present TriShield, a three-layer deterministic defense that completely prevents NeuroImprint-style reconstruction with zero model utility loss and no additional communication rounds. TriShield consists of: (1) a Parameter Artifact Detector that identifies
memory-
neuron signatures in distributed model parameters before local training begins; (2) a Stateful Virtual Iteration} mechanism that forces Adam/AdamW's momentum state to irreversibly entangle gradients across virtual steps, invalidating NeuroImprint's closed-form inversion; and (3) a Zero-Utility Orthogonal Projection operator that projects all local gradient updates onto the main-task semantic subspace computed via SVD, physically eliminating any gradient components that carry private memorization. We prove theoretically that after Layers 2 and 3, the mutual information between the uploaded gradient and any individual training sample is zero. Experiments on GPT-2 (117M) and Llama-Guard-3-1B verify that TriShield reduces NeuroImprint reconstruction rate to 0% across all tested attack variants, while maintaining or improving training accuracy, with less than 5% additional GPU computation overhead.
Read the full abstract ▾