Computation and Language
Forecasting With LLMs: Improved Generalization Through Feature Steering
Humzah Merchant and Bradford Levy
Successful
forecasting involves identifying patterns between historical and future states of the world which generalize to future observations. We apply LLMs to a variety of
forecasting tasks and inspect their internal states using sparse autoencoders to understand whether they appear to rely on time-specific pieces of knowledge versus generalizable patterns. Our analyses identify features associated with both time-aware
reasoning and look-ahead-biased
reasoning. We then apply the LLMs to an entirely different domain and intervene on these features. We find that amplifying time-awareness features substantially reduces look-ahead bias on
forecasting prompts while preserving general
reasoning performance. In contrast, steering the candidate look-ahead-bias features does not produce an effect. These results suggest that interpretable temporal features can be used to causally shift LLMs toward more historically grounded
reasoning.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models
Jiajun Wu, Haoyu Kang, Yining Sun et al. (+13 more)
Large
vision-
language models (LVLMs) have recently shown immense potential in automated content moderation, sparking growing interest in developing harmful-
video benchmarks. However, we identify two primary limitations in existing works: 1) The multi-layered characteristics of harmful
videos are overlooked. Existing
benchmarks predominantly formulate evaluation as a binary classification task, failing to capture implicit or deep contextual harms. 2) Explanatory rationales are completely absent. Current frameworks measure exclusively whether a model flags a
video correctly rather than explaining why, turning evaluation into a black box where models can succeed through superficial shortcuts. To address these problems, we present HarmVideoBench, a multi-layered diagnostic
benchmark comprising 1,379
videos paired with 4,137 multiple-choice questions. HarmVideoBench
benchmarks three hierarchical dimensions: Observable Evidence, Clip-Internal Meaning, and Beyond-Clip
Reasoning, aiming to evaluate models' deep understanding beyond surface cues with carefully balanced and curated samples. We evaluate 19 leading models on HarmVideoBench to assess their multidimensional understanding of harmful
videos. Moreover, we introduce BCR, a
benchmark-aligned method that predicts
reasoning boundaries and dynamically retrieves context only when needed. Experimental results show that BCR substantially improves the base model's performance in harmful
video understanding, raising the macro average from 61.7 percent to a state-of-the-art 84.4 percent.
Read the full abstract ▾
All authors Jiajun Wu, Haoyu Kang, Yining Sun, Jiacheng Hou, Heng Zhang, Danyang Zhang, Zhenjun Zhao, Haochi Zhang, Leixin Sun, Eric Hanchen Jiang, Yushan Li, Ruiyu Li, Mengkai Huang, Yan Gao, Xu Zhang, Guancheng Wan
Computation and Language
The Riddle Riddle: Testing Flexible Reasoning in Large Language Models and Humans
Bella Fascendini, Kathryn McGregor, Max D. Gupta et al. (+1 more)
Humans flexibly adapt their
reasoning strategies to the requirements of a given problem. Large
language models (LLMs) have performed well on many cognitive tasks, however, it is unclear whether this accuracy is a result of pattern matching from training data or flexible
reasoning. Here, we introduce a novel paradigm to test this question: the riddle riddle paradigm. Riddle riddles are word problems written to mimic popular riddles, but altered so their answers only require literal interpretations. Identifying correct answers requires looking past the structure of each question and flexibly apply different
reasoning strategies based on the content. If LLMs respond to surface features, such as form, a riddle-like structure should cause models to use an inventive
reasoning strategy even when a literal interpretation suffices. Alternatively, if LLMs reason based on content, they should flexibly switch strategies when appropriate. Across two experiments with nine state-of-the-art LLMs and 100 human participants, we show humans and LLMs fail on this paradigm in opposite directions. LLMs were far more accurate on genuine riddles than on riddle riddles (84.9% vs. 50.7%); whereas humans showed the reverse effect (50.5% vs. 80.5%). Error analysis shows that 90.8% of LLM errors on riddle riddles (the condition where they show diminished performance) were due to inappropriate use of inventive
reasoning while only 57.6% of human errors on genuine riddles were due to overextending literal
reasoning. Thus, while both groups make mistakes,
reasoning mistakes are made more often by LLMs than by humans. Overall, LLMs' strong performance on genuine riddles may reflect
memory retrieval rather than flexible strategy selection, and without stimuli designed to elicit this contrast, it becomes easy to conflate LLM-generated outputs that look like
reasoning with genuine
reasoning.
Read the full abstract ▾
All authors Bella Fascendini, Kathryn McGregor, Max D. Gupta, Thomas L. Griffiths
Computation and Language
Towards Explainable Adjudicative Variance: Quantifying Judicial Discretion via Gated Multi-Task Learning
Stanisław Sójka, Felix Steffek and Matthias Grabmair
Legal outcome prediction must disentangle objective case facts from adjudicative context. Merit-based rulings rely on factual evidence while technical disposals may hinge on judicial discretion. We propose a Judge-Aware Gated Multi-Task Learning architecture that explicitly models this distinction. We introduce a fine-grained outcome taxonomy to supervise the encoder, enforcing a structural regularization that disentangles distinct semantic pathways. This granular legal curriculum enables our Gated Fusion mechanism to dynamically modulate reliance on judge identity. We evaluate our approach on 13,937 UK Employment Tribunal decisions. We
benchmark our design against supervised fine-tuning (SFT) of a Gemma-4 26B-A4B backbone, in which judge identity and the taxonomy are injected as
prompt tokens or autoregressive output targets. The two contextual signals compose only weakly when forced through a single autoregressive channel. In contrast, coupling a LoRA-adapted Gemma-4 encoder with our gated architecture defines a new state of the art on this
benchmark while requiring an order of magnitude fewer trainable parameters than the generative SFT baselines, with gains concentrated on the most ambiguous and rarest outcome classes. Beyond accuracy, the architecture is interpretable; learned judge
embeddings and calibration profiles localize the cases where adjudicative context drives the prediction. These results indicate that, for identity-conditioned classification of legal outcomes, the choice of conditioning interface dominates scale: differentiable structured composition yields more accurate, more parameter-efficient models than
prompt-based composition over a substantially larger backbone.
Read the full abstract ▾
Computation and Language
NuclearQAv2: A Structured Benchmark for Evaluating Domain-Science Competence in Large Language Models
Henry Shaowu Yuchi, Michal Kucer, Benjamin H. Sims et al. (+2 more)
Large
language models (LLMs) have demonstrated strong performance across a wide range of tasks, but ensuring their reliability in highly technical domains remains a significant challenge. In
nuclear engineering, problem solving often requires not only factual knowledge but also quantitative
reasoning and conceptual understanding. To address the need for systematic evaluation in this domain, we introduce NuclearQAv2, a
benchmark for assessing LLMs on
nuclear engineering knowledge. The
benchmark comprises approximately 1,240 question-answer pairs spanning three categories: boolean, numeric, and verbal. NuclearQAv2 is constructed using a hybrid pipeline that combines expert-authored questions, existing
datasets, and LLM-assisted generation from domain-specific technical corpora. By leveraging structured prompting for both automated question generation and response evaluation, the proposed framework enables scalable
benchmark construction and evaluation. We evaluate a diverse set of LLMs using NuclearQAv2 and observe substantial performance differences across task types. While the models generally perform well on factual questions, quantitative
reasoning and conceptual understanding remain considerably more challenging. These results highlight the importance of multi-faceted evaluation frameworks and establish NuclearQAv2 as a scalable
benchmark for assessing LLM capabilities in technical domains.
Read the full abstract ▾
All authors Henry Shaowu Yuchi, Michal Kucer, Benjamin H. Sims, Selma Peterson, Emily Taylor
Computation and Language
Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
Zhenhua Xu, Dongsheng Chen, Jian Li et al. (+7 more)
Building general-purpose role-playing
agents that faithfully portray any character from a natural-
language profile remains challenging. The dominant paradigm – supervised fine-tuning – encourages behavioral mimicry without deep, human-like internal thought processes, resulting in poor out-of-distribution generalization. Therefore, we propose Psy-CoT, a psychology-grounded chain-of-thought framework that decomposes pre-response
reasoning into three role-specific steps – Interaction Perception, Psychological Empathy, and Logical Construction – so that the model thinks dynamically from the profile rather than merely mimicking surface patterns. While structured
reasoning provides a foundation, it alone is insufficient; reinforcement learning is essential to further align the model with character fidelity. However, we observe that under LLM-based reward models, both generic phrases that hack the reward model and genuinely role-specific phrases receive identical gradient signals – this hacking accumulates over training, misleading the model into treating both as equally optimal choices. To address this, we propose Role-Aware Policy Optimization (RAPO), which uses profile–
token mutual information to weight gradients asymmetrically – amplifying role-specific
tokens under positive advantage while attenuating them under negative advantage. Experiments on CoSER, CharacterBench, and CharacterEval demonstrate that Psy-CoT outperforms existing role-playing CoT methods, and RAPO consistently surpasses GRPO across multiple model scales.
Read the full abstract ▾
All authors Zhenhua Xu, Dongsheng Chen, Jian Li, Yitong Lin, Zhebo Wang, Jiafu Wu, Yizhang Jin, Chengjie Wang, Meng Han, Yabiao Wang
Computation and Language · Cross-listed from cs.LG
Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA
Eren Senoglu, Federico Toschi, Nicolo Brunello et al. (+2 more)
Multimodal large
language models (MLLMs) applied to Medical Visual Question Answering (VQA) tend to produce overconfident outputs regardless of actual correctness, and existing verbalized confidence calibration methods, developed primarily for text only LLMs, do not account for the
multimodal nature of medical image understanding. This work proposes a training based framework that finetunes MLLMs to improve their calibration using a composite loss function combining a Brier style calibration term, an anchor regularizer that prevents confidence collapse toward extreme values, a contrastive image text
alignment term, and a KL based model stabilization term. The
alignment signal is derived from a $2 \times 2$ factorial perturbation design that crosses image presence with text integrity, probing the reliance of the model on visual modality input versus
language priors. Finally, a top K KL divergence regularizer is used to protect the answering ability of the model during finetuning. Across three Medical VQA
benchmarks and two architectures (MedGemma 4B IT and Qwen2 VL 7B Instruct), our method reduces calibration error by 60% or more, and improves discrimination by 26% or more, while preserving predictive accuracy. On average across
benchmarks, the technique outperforms prompting based, sampling based, and training based approaches, and ablation experiments confirm that each component of the loss function is indeed necessary for improving the calibration. All code for the experiments is publicly available.
Read the full abstract ▾
All authors Eren Senoglu, Federico Toschi, Nicolo Brunello, Andrea Sassella, Mark James Carman
Computation and Language
MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment
Sander Land
The Unigram tokenizer uses an elegant representation which makes it straightforward to edit vocabularies, but its training is comparatively heavy and complex. We introduce MinGram (Minimalist Unigram), which keeps the
token-list representation but simplifies training using a BPE-derived seed vocabulary, Hard EM on a minimum-
token path, and a single flat score-
pruning step. This removes the suffix array, the forward-backward pass, and the iterative prune loop, leaving a procedure that requires little beyond tokenizer inference itself. By making
token count the primary objective and using a Unigram score only as a tiebreak, MinGram keeps the compression of pure
token-count methods while retaining much of the morphological
alignment and downstream quality of probabilistic ones. Across six
languages, MinGram compresses better than both BPE and standard Unigram, and a compression-oriented variant matches the strongest
token-count compressors while retaining substantially higher morphological
alignment. In controlled downstream
language-model training, Unigram-family tokenizers, with MinGram among the best, consistently beat BPE in bits-per-byte.
Read the full abstract ▾
Computation and Language
Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs
Sinie van der Ben, Raphaël Baur, Yannick Metz et al. (+1 more)
Recent work identified
emotion vectors in Claude Sonnet 4.5, which are internal representations that encode
emotion concepts, causally influence behavior, and exhibit geometry mirroring human psychological structure. We test the generality of these findings in two open-weight models, Apertus-8B-Instruct-2509 and Gemma-4-E4B-it, extracting
emotion contrast vectors across all layers, using two model-generated corpora. We recover valence geometry for both models, with peak PC1–valence correlations of $r = 0.76$ and $r = 0.83$, approaching the $r = 0.81$ reported for Claude.Beyond replication, we observe notable differences in how valence representations emerge across model depth. In Gemma-4-E4B-it, valence is strongly encoded in early layers but collapses towards later layers, whereas Apertus-8B-Instruct-2509 exhibits the opposite pattern, with valence representations absent in early layers, but emerging at mid depths. Arousal encoding, in contrast, is sensitive to the extraction corpus: both models show stronger PC2–arousal
alignment with Gemma-generated stories ($r$ up to $0.45$) than Apertus-generated ones ($r \leq 0.21$), suggesting arousal-relevant cues are unevenly distributed across generated corpora. We open-source our experiment code and
dataset for reproducible investigation of
emotion representations across
language model architectures.
Read the full abstract ▾
All authors Sinie van der Ben, Raphaël Baur, Yannick Metz, Mennatallah El-Assady
Computation and Language
ReaORE: Reasoning-Guided Progressive Open Relation Extraction Empowered by Large Reasoning Models
Xin Lin, Liang Zhang, Guoqi Ma et al. (+2 more)
Open Relation Extraction (OpenRE) requires a model to extract unseen relations between head and tail entities from unstructured text for real-world applications. The core challenge of OpenRE lies in achieving reliable generalization to unseen relation types. Current OpenRE approaches either employ clustering techniques, which cannot generate relation labels and suffer from poor generalization, or rely on direct relation label generation via Large
Language Models (LLMs), which lack sufficient discriminative capacity to distinguish easily confused relations. To address these limitations, we propose
Reasoning-guided progressive OpenRE (ReaORE), a framework for performing relation extraction through coarse-to-fine relation
reasoning. Specifically, ReaORE consists of two key stages: (i) relation filtering, which reasons over multiple aspects to understand relations and instances, yielding an initial relation set, and further supplements and filters relations via
embedding-based similarity to ensure the target relation is included; (ii) relation prediction, which aims to predict the target relations from the above set via fine-grained comparative
reasoning to better distinguish easily confused relations. Extensive experiments on two widely used OpenRE
datasets demonstrate that ReaORE outperforms existing baselines.
Read the full abstract ▾
All authors Xin Lin, Liang Zhang, Guoqi Ma, Hongyao Tu, Jinsong Su
Computation and Language
Auditing Framing-Sensitive Behavioral Instability in Large Language Models for Mental Health Interactions
Abla Bedoui, Ashley L. Greene and Mohammed Cherkaoui
Large
language models (LLMs) are increasingly being integrated into mental health support tools and other psychologically sensitive conversational applications. In such settings, behavioral stability and consistency are important for trustworthy human-AI interaction. However, semantically similar concerns can be presented through different contextual framings, potentially eliciting different model responses. Such framing-sensitive variability may challenge user expectations regarding system behavior and complicate the assessment of AI reliability. While prior studies have primarily examined such effects at the behavioral level, less is known about how framing-related variation is reflected in the internal representations of aligned
language models. In this work, we investigate these effects using controlled matched
prompts spanning multiple contextual framing conditions across several instruction-tuned model families. Across architectures, framing systematically alters interpretive response tendencies. Layer-wise probing analyses show that behavior-associated information remains decodable throughout
transformer depth, with architecture-dependent variation in decoding strength. Moreover, held-out framing probes remained consistently above chance across architectures despite strong lexical baselines. Activation steering experiments further suggest that framing-associated representational directions can partially modulate downstream behavioral outcomes. Finally, these findings indicate that robustness to contextual variation may represent an important consideration when evaluating the consistency and trustworthiness of conversational AI systems deployed in mental-health-oriented interactions.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Einstein World Models
Munachiso Samuel Nwadike, Zangir Iklassov, Ali Mekky et al. (+2 more)
Does intelligence require the ability to reason about phenomena beyond direct experience? It is natural to suspect that some complex thought cannot be captured through
language alone. However, of particular concern to this work, is whether visualising counterfactual events can complement
language as a mechanism for complex thought. We ask whether LLMs can be trained to utilise such visualisation mechanisms, in a way that benefits their
reasoning abilities. Motivated by this question, we propose Einstein World Models. EWMs are a blueprint for LLM-based
reasoning systems that place visual-temporal rollouts inside the
reasoning trace, allowing them to reason in ways that text alone may not support well. In an EWM, the LLM calls a world-module (not to be confused with a world model), to produce short rollouts of scenes under consideration. The returned rollout is treated not as the answer, but as an inspectable hypothesis that can support later
reasoning. Einstein World Models extend the capability of LLMs for tool calling (such as web search or code execution), into the domain of visual thought experiments.
Read the full abstract ▾
All authors Munachiso Samuel Nwadike, Zangir Iklassov, Ali Mekky, Zayd M. Kawakibi Zuhri, Kentaro Inui
Computation and Language
Beatrice Savoldi, Sara Papi, Wafa Aissa et al. (+2 more)
Speech-capable models are increasingly deployed in real-world applications across
languages. Yet their safety and
fairness beyond English settings and under naturalistic conditions remain understudied. We survey safety reporting practices across state-of-the-art
speech model releases, finding that only 8% document any
multilingual analysis. To address this gap, we introduce RedVox, a
multilingual safety and
fairness benchmark for audio and
speech built on real voices, covering unsafe and unfair stereotypical requests across five
languages (English, French, Italian, Spanish, and German). Evaluating eight state-of-the-art models, we find that vulnerabilities persist even under non-
adversarial conditions, worsen in non-English
languages, and are amplified when the request comes from a spoken input. Finally, by surveying the participants who contributed to RedVox, we document the unique personal and
privacy challenges of collecting
speech data with human participants, pointing to broader sociotechnical challenges in naturalistic
speech safety research.
Read the full abstract ▾
All authors Beatrice Savoldi, Sara Papi, Wafa Aissa, Matteo Negri, Luisa Bentivogli
Computation and Language
Term-Centric Hierarchy Induction from Heterogeneous Corpora
Elena Senger, Yuri Campbell, Jan-Peter Bergmann et al. (+2 more)
Organizing knowledge from diverse text sources into interpretable hierarchies is crucial for tasks such as policy analysis, innovation monitoring, and exploratory domain mapping. Existing taxonomy induction methods typically rely on document-level representations that capture entire documents rather than the specific domain concepts relevant for knowledge organization, limiting their ability to generalize across heterogeneous sources. We propose a term-centric framework for inducing hierarchical taxonomies from heterogeneous corpora that scales to massive document collections. Our approach maps documents from diverse sources into a shared representation space using automatic term extraction, enabling robust cross-source
alignment. Based on these representations, we construct interpretable hierarchies that integrate domain priors with datadriven clustering. Experiments on a novel English and German multi-source
benchmark of over one million documents demonstrate that our method improves cross-source coherence and hierarchy quality over text- and summarybased baselines. A case study on German regional innovation analysis further demonstrates its practical utility for technology landscape mapping.
Read the full abstract ▾
All authors Elena Senger, Yuri Campbell, Jan-Peter Bergmann, Rob van der Goot, Barbara Plank
Computation and Language · Cross-listed from cs.CR
Jailbreaking for the Average Jane: Choosing Optimal Jailbreaks via Bandit Algorithms for Automatically Enhanced Queries
Prarabdh Shukla, Ritik, Suhas Rao et al. (+2 more)
With a profusion of jailbreaks for LLMs now widely known, a growing concern is that non-expert malicious actors ("the average Jane") could elicit actionable responses to malicious requests. In this work, we examine whether this concern is justified. A non-expert malicious actor requires two ingredients for a successful attack: a powerful jailbreak for their target model, acting on an effective malicious query. For the former, we propose a novel attack strategy based on the multi-armed bandit framework. This allows efficient online learning of the optimal jailbreak from a large choice set via noisy exploration on a small number of queries, with subsequent application of the learnt policy on an exploitation set. For the latter, we curate $\mathrm{FrankensteinBench}$, a safety
benchmark of $11,279$ malicious queries drawn from manual curation over $7$ existing
benchmarks, along with automated enhancement and generation. Each query is categorized as simple or complex by the technical expertise required to craft it. Our findings confirm the concern. Our bandit-based attack achieves success rates as high as $97\%$ on average over $15$ SoTA open-weight LLMs. Moreover, adding complexity to queries raises the attack success rate by up to $26\%$ on average across models – making it an effective, automatable prompting strategy.
Read the full abstract ▾
All authors Prarabdh Shukla, Ritik, Suhas Rao, Arpit Agarwal, Arjun Bhagoji
Computation and Language
GAVEL: Grounded Caption Error Verification and Localization
Zixian Gao, Atsushi Hashimoto and Kuniaki Saito
Vision-
language models (VLMs) often produce hallucinated or inconsistent outputs, where text and images are not properly aligned. Addressing this issue requires not only detecting misalignment but also explaining the discrepancy and localizing its visual evidence. We introduce GAVEL (Grounded Caption Error Verification and Localization), a task that jointly addresses verification, explanation, and localization for image-text pairs. To support systematic evaluation, we also present a corresponding
dataset and
benchmark. We further train a supervised baseline on the human-annotated training split to assess whether GAVEL provides learnable supervision for these abilities. Experiments show that even strong closed-source models struggle on GAVEL, while the supervised baseline yields consistent improvements across grounding and explanation metrics.
Read the full abstract ▾
Computation and Language
Subham Kumar, Prakrithi Shivaprakash, Abhishek Manoharan et al. (+7 more)
Automatic
Speech Recognition (ASR) is increasingly used to document clinical encounters, yet its reliability in
multilingual and demographically diverse Indian healthcare context remains largely unknown. In this study, we first conduct the systematic audit of ASR performance on real-world psychiatric interview data spanning Kannada, Hindi and Indian English, comparing eight state-of-the-art models including IndicWhisper, WhisperLargeV3, Sarvam, GoogleS2T, Gemma3n, OmniLingual, Vaani, and Gemini. Our results reveal substantial variability across models and
languages, with some systems performing competitively in Indian English but failing in regional
speech. We further fine-tune two of the best performing opensource models, i.e., Gemma3n and OmniLingual, using various methods. With this, we uncover systematic performance gaps tied to speaker role and gender, raising concerns about equitable deployment in clinical settings, which are further mitigated by
fairness-aware fine-tuning. To this end, we propose SamaVaani, a unified debiasing technique that simultaneously improves ASR performance and improves
fairness across demographic groups.
Read the full abstract ▾
All authors Subham Kumar, Prakrithi Shivaprakash, Abhishek Manoharan, Astut Kurariya, Diptadhi Mukherjee, Prabhat Chand, Pratima Murthy, Koustav Rudra, Lekhansh Shukla, Animesh Mukherjee
Computation and Language
Heterogeneous Neural Predictivity from Language Models During Naturalistic Comprehension
Xiao Jia
Language-model representations provide structured, high-dimensional annotations of naturalistic
language stimuli and can serve as informative neural predictors during comprehension. We analyzed locked derived data from
Brain Treebank, MEG-MASC, and Podcast ECoG with eight frozen
language models, blocked encoding models, and matched temporal, nuisance, and representation-capacity controls. Positive held-out prediction and gains over low-level baselines were widespread in source-level summaries. Across
Brain Treebank and Podcast ECoG, 67 of 432 evaluable rows met a controlled predictive-only criterion, and model-side feature ablations changed prediction scores in most evaluable source rows.
Brain-derived, timing-linked, acoustic, and implanted-signal controls confirmed component-level sensitivity of the analysis pipeline. These findings show that
language-model-derived quantities can annotate neural activity during natural
speech and text comprehension. Participant-level matched-control advantages were localized rather than uniform, response-profile and feature-specificity contrasts bounded representational or computational interpretations, and complete co-indexed integrated interpretation will require future jointly indexed coverage. Together, the analyses identify
language-model features as useful neural predictors and separate predictive usefulness from claims about shared neural organization or
language-processing computations.
Read the full abstract ▾
Computation and Language
Information-Aware KV Cache Compression for Long Reasoning
Jushi Kai, Zhuiri Xiao, Alexandra Birch et al. (+1 more)
Reasoning capability has advanced rapidly in large
language models (LLMs), leading to an increasing size of key-value (KV) cache in both prefilling and decoding stages. Existing KV cache compression methods mainly rely on
attention weights to estimate
token importance. While
attention effectively captures contextual relevance, it overlooks complementary information-theoretic signals related to predictive uncertainty and
token informativeness. In this paper, we revisit
token importance from a forward-looking perspective and introduce Forward Influence, a metric that measures how compressed
tokens affect future contexts. Our analysis reveals that
tokens selected by
attention scores mainly influence nearby contexts, whereas
tokens associated with high predictive uncertainty exhibit substantially stronger influence on distant future contexts. Based on the observation, we propose InfoKV, an
entropy-aware KV cache compression framework that incorporates information-theoretic signals. It combines
token-level predictive uncertainty with layer-wise representation
evolution and integrates the resulting
entropy scores with
attention scores during
reasoning. Experiments on long-context
reasoning benchmarks with Llama-3.1, Llama-3.2, and DeepSeek-R1 demonstrate that InfoKV consistently outperforms existing
attention-based KV compression methods in both long prefilling and decoding scenarios.
Read the full abstract ▾
All authors Jushi Kai, Zhuiri Xiao, Alexandra Birch, Zhouhan Lin
Computation and Language
Cascaded Multi-Granularity Pruning for On-Device LLM Inference in Industrial IoT
Jinghan Wang, Yanjun Chen, Wei Zhang et al. (+3 more)
Deploying large
language models (LLMs) on Industrial Internet of Things (IIoT) edge devices demands extreme compression, yet existing structured
pruning methods collapse at high compression ratios due to one-shot importance estimation, and their cross-architecture behavior remains unpredictable. This article presents a cascaded multi-granularity
pruning framework that removes layers,
attention heads, and feed-forward channels in coarse-to-fine order, with lightweight low-rank recovery between stages to re-estimate component importance. An information-theoretic analysis motivates this ordering, and the Structural Independence Assumption (SIA) is formalized as a checkable condition predicting whether per-component
pruning criteria are reliable for a given architecture: Multi-Head
Attention (MHA)+GELU designs satisfy the SIA, whereas Grouped Query
Attention (GQA)+SwiGLU designs violate it. On bearing fault diagnosis spanning 88M to 6.25B-parameter models, the framework extends achievable compression to 13.8 times on MHA+GELU architectures with 83.82% accuracy (+3.70 percentage points (pp) over the strongest baseline), while exposing a ~74pp accuracy collapse on GQA+SwiGLU architectures that violate the SIA. Deployed on an industrial slewing bearing fault diagnosis platform with NVIDIA DGX Spark, compressed models reduce inference latency by up to 67.2% and peak
memory by 62.5%, demonstrating viability for IIoT edge inference.
Read the full abstract ▾
All authors Jinghan Wang, Yanjun Chen, Wei Zhang, Xiaotong Huang, Tianchen Liu, Gaoliang Peng
Computation and Language · Cross-listed from cs.AI
AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems
Changxin Lao, Fei Pan, Guozhuang Ma et al. (+59 more)
Recommendation
algorithm iteration is moving from an artisanal, engineer-bound process toward an industrialized research loop, but this transition remains blocked by a structural execution bottleneck: the idea-to-launch cycle still depends on human engineers to generate hypotheses, modify production code, launch A/B experiments, and attribute online results. Innovation therefore scales linearly with headcount rather than compounding with evidence, compute, and accumulated experimental knowledge. We present AgentX, a production-deployed multi-
agent system that fundamentally restructures this production function. AgentX operates as a self-evolving development engine: it autonomously generates, implements, evaluates, and learns from recommendation experiments at a scale and pace that no manual workflow can sustain. The system orchestrates four tightly coupled stages in a closed loop. A Brainstorm
Agent synthesizes evidence from historical experiments, system architecture, data analysis, and external research into ranked, executable proposals. A Developing
Agent translates each proposal into production-ready code through repository-grounded generation and multi-dimensional reliability verification. An Evaluation
Agent conducts safe online rollout with guardrail-vetoed A/B judgment, converting both successes and failures into structured knowledge assets. A Harness
Evolution layer (SGPO) then distills execution trajectories into semantic-gradient updates that continuously sharpen the
agents themselves – making the system not merely automated, but self-improving.
Read the full abstract ▾
All authors Changxin Lao, Fei Pan, Guozhuang Ma, Han Li, Huihuang Lin, Jijun Shi, Kangzhi Zhao, Kun Gai, Mo Zhou, Qinqin Zhou, Quan Chen, Ruochen Yang, Shifu Bie, Shijie Yi, Shuang Yang, Shuo Yang, Wenhao Li, Wentao Xie, Xiao Lv, Xuming Wang, Yijun Wang, Yiming Chen, Yusheng Huang, Zhongyuan Wang, Zibo Zhao, Zijie Zhuang, Baoning Xia, Chao Liu, Chaoyi Ma, Chubo He, Dawei Cong, Feng Jiang, Gang Wang, Guilin Xia, Hanwen Xu, Jiahong Xie, Jiahui Qiao, Jian Liang, Jiangfan Yue, Jing Wang, Jinghan Yang, Jinghui Jia, Kan Qin, Lei Wang, Ming Li, Peilin Song, Pengbo Xu, Qiang Luo, Ruiming Tang, Shiyang Liu, Shuxian Jin, Tao Wang, Tao Zhang, Xiang Gao, Xianghan Li, Yingsong Luo, Yiwen Ning, Yongcheng Liu, Yueyang Liu, Yuan Guo, Zhaojie Liu, Zhenkai Cui
Computation and Language
FBK's Long-form SpeechLLMs for IWSLT 2026 Instruction Following
Zhihang Xie, Marco Gaido, Sara Papi et al. (+2 more)
This paper describes our submission to the IWSLT 2026 Instruction Following shared task. SpeechLLMs are developed for both short-form and long-form
speech instruction following under constrained settings. For the short track, strong performance is achieved on MCIF, with a SIFS score of 2.0708. For the long track, three
speech segmentation methods are explored, and the HIFS score is introduced to account for unstable long-form generation. Experimental results show that fixed 30-second
segmentation provides the most robust long-form performance, achieving the highest HIFS score of 2.0663. Further analysis shows that
hallucination mainly manifests as repetitive insertions in generated outputs, substantially affecting ASR and SSUM, while short-form capabilities are largely retained after long-form extension.
Read the full abstract ▾
All authors Zhihang Xie, Marco Gaido, Sara Papi, Matteo Negri, Luisa Bentivogli
Computation and Language · Cross-listed from cs.AI
KARLA: Knowledge-base Augmented Retrieval for Language Models
Francois Crespin, Fabian M. Suchanek and Nils Holzenberger
We propose a new method that allows an LLM to automatically pull in factual knowledge from a knowledge base during
token generation. This means that (1)~factual knowledge in the LLM output can be updated without retraining the LLM, (2)~facts in the LLM output can be traced to the knowledge base for transparency and explainability, and (3)~smaller models can achieve the same factual accuracy as larger models. Our core idea is to train the model to produce special
tokens that trigger a query to the knowledge base. Our experiments show that our method improves factual grounding in both short and long-form generation, and allows factual revisions to take effect through KB edits rather than parameter updates.
Read the full abstract ▾
Computation and Language
From Vajrayana Tara to Bengali Baul: A Computational Study of Lexical Transmission Across Buddhist, Shakta, and Vaishnava Traditions in Bengal
Joy Bose
We present a computational corpus study of vocabulary relationships across eight tradition layers of Bengali and Sanskrit devotional literature spanning the 8th to 19th centuries, encompassing Buddhist Vajrayana, Shakta Tantra, Vaishnava, and Baul traditions. Using a corpus of 75 texts and TF-IDF character n-gram vectorization with cosine similarity analysis, we address the historically argued but previously unquantified claim that Buddhist Vajrayana vocabulary survived the collapse of the Pala monasteries and was absorbed into the Shakta Tantra tradition of Bengal. The central finding is a specificity result: the Gitagovinda (Vaishnava Sanskrit, 12th century) has zero cosine similarity to Shakta Kali texts, while Bridge Tara texts (Buddhist-Shakta transitional, same century, same
language) have cosine similarity 0.54 to Shakta Kali. This 8.5-fold contrast between two Sanskrit traditions from the same century demonstrates that the Buddhist-Shakta vocabulary overlap is not a generic property of Sanskrit devotional literature but is specific to the Buddhist-Shakta transmission chain. Three Brihannilatantra Tara texts show Shakta-to-Buddhist vocabulary ratios of 2.0 to 4.0, constituting measurable evidence of lexical transition within that chain. Ramprasad Sen's 18th-century Bengali Kali songs preserve Buddhist vocabulary residue including 56 occurrences of Tara alongside 103 occurrences of Kali. The Vaishnava Bengali tradition contributes a parallel chain to modern Baul vocabulary (similarity 0.29), slightly weaker than the Buddhist Sahajiya chain via Charyapada (0.31). These results provide the first quantitative multi-tradition corroboration of historically argued Buddhist-Shakta syncretism in Bengal.
Read the full abstract ▾
Computation and Language
OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning
Shuo Yang, Jinyang Wu, Zhengxi Lu et al. (+8 more)
Outcome-based reinforcement learning provides a stable optimization backbone for
language agents, but its sparse trajectory-level rewards provide little guidance on which intermediate decisions should be reinforced or suppressed. On-policy self-
distillation offers dense
token-level supervision, yet existing skill-conditioned variants often rely on external skill
memories or retrieved privileged context, which are costly to maintain and can be mismatched with the state distribution induced by the current policy in multi-turn interaction. We propose OPID (On-Policy Skill
Distillation), a framework that extracts skill supervision directly from completed on-policy trajectories. OPID represents trajectory hindsight as hierarchical skills: episode-level skills capture global workflows or failure-avoidance rules, while step-level skills capture local decision knowledge at critical timesteps. A critical-first routing mechanism uses step-level skills when critical decisions are identified and falls back to episode-level skills as default guidance otherwise. The selected skill is injected into the interaction history, allowing the old policy to re-score the same sampled response under both original and skill-augmented contexts. The resulting log-probability shift yields a
token-level self-
distillation advantage, which is combined with the outcome advantage for policy optimization. OPID thus preserves RL as the primary training objective while introducing dense, distribution-matched hindsight supervision. Experiments on ALFWorld, WebShop and Search-based QA demonstrate that OPID generally improves
agent performance, sample efficiency, and robustness over outcome-only RL and existing skill-
distillation baselines. Our code is available at https://github.com/jinyangwu/OPID/tree/main.
Read the full abstract ▾
All authors Shuo Yang, Jinyang Wu, Zhengxi Lu, Yuhao Shen, Fan Zhang, Lang Feng, Shuai Zhang, Haoran Luo, Zheng Lian, Zhengqi Wen, Jianhua Tao
Computation and Language · Cross-listed from cs.LG
AIGP: An LLM-Based Framework for Long-Term Value Alignment in E-Commerce Pricing
Chennan Ma, Yanning Zhang, Siqi Hong et al. (+3 more)
Traditional dynamic pricing models in large-scale e-commerce suffer from limited interpretability, poor utilization of unstructured information, and misalignment with long-term business objectives such as cumulative Gross Merchandise Value (GMV), Return on Investment (ROI) and milestone achievement. We propose AIGP, a novel framework that leverages a Large
Language Model (LLM) prompted with domain knowledge, structured data and textual context to make interpretable, knowledge-aware pricing decisions. For efficient deployment while maintaining high-quality outputs, we employ supervised fine-tuning for knowledge
distillation. Central to AIGP is the Long-Term Value
Estimator (LTVE), trained via offline reinforcement learning on historical data, which serves as a reward model to score candidate pricing actions and select preference pairs for Direct Preference Optimization (DPO), thereby aligning the pricing policy with long-term business objectives. Extensive offline evaluations and large-scale online A/B tests on Tao Factory demonstrate that AIGP achieves significant improvements: +13.21% in GMV, +7.59% in ROI, and +8.20% in milestone achievement rate over 14 days compared to the production baseline, while simultaneously providing interpretable and transparent pricing rationales.
Read the full abstract ▾
All authors Chennan Ma, Yanning Zhang, Siqi Hong, Xiuchong Wang, Fei Xiao, Keping Yang
Computation and Language · Cross-listed from cs.LG
Reproducibility Study of "AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models"
Ananth K Suresh and Arya Hariharan
Fang et al. (2025) introduced a null-space constrained projection, named AlphaEdit, for locate-then-edit knowledge editing methods, theoretically guaranteeing that edits do not disrupt previously preserved knowledge, and reports substantial gains over existing editing methods on LLaMA3, GPT2-XL, and GPT-J. In this work, we present a reproducibility study of AlphaEdit, reproducing its reported results under the original experimental setup and extending the evaluation along three axes: new model architectures, additional downstream
benchmarks, and substantially longer sequential editing horizons. We successfully reproduce AlphaEdit's reported metrics across the original models, though we identify a discrepancy in the reported fluency and consistency metric. Extending AlphaEdit to newer model families, we find that its advantage does not generalize uniformly, which we trace to architectural assumptions in the locate-then-edit paradigm that are violated by these newer models. We further stress-test AlphaEdit's central sequential-editing claim by extending the number of edits well beyond those evaluated in the original paper, and find that performance, which is stable at the originally reported scale, degrades as edits reach a much higher count, indicating that the null-space projection's protection against catastrophic forgetting is bounded rather than unconditional. Finally, we extend evaluation of edited models on three extra
benchmarks, namely, BoolQ, HellaSwag, and XSTest, and we find that large-scale sequential editing degrades both general downstream task competence and safety-relevant refusal behavior. Our results confirm that AlphaEdit performs as reported within its original scope, while showing that its core theoretical guarantees are sensitive to model architecture and editing scale in ways that have practical implications for its deployment.
Read the full abstract ▾
Computation and Language
Evaluation Pitfalls and Challenges in Multimedia Event Extraction
Philipp Seeberger, Steffen Freisinger, Tobias Bocklet et al. (+1 more)
Multimedia event extraction aims to jointly identify events and their arguments across multiple modalities, such as text and images, to support more comprehensive event understanding. While recent work reports steady and substantial progress, the reliability and comparability of these results critically depend on consistent and rigorous evaluation. In this work, we present the first systematic analysis of evaluation pitfalls in multimedia event extraction and identify three major sources of issues: inconsistent data processing, inconsistent task assumptions, and overly relaxed evaluation settings. We demonstrate, through a series of controlled experiments under a strict evaluation framework, that minor evaluation choices can cause large performance variations and lead to overestimation of a model's ability to ground real-world events across modalities. Our findings highlight the need for comparable evaluation standards and encourage a shift toward more rigorous evaluation in multimedia event extraction.
Read the full abstract ▾
All authors Philipp Seeberger, Steffen Freisinger, Tobias Bocklet, Korbinian Riedhammer
Computation and Language
ConvMemory v3: A Validity Context Layer for Conversational Memory via Target-Conditioned Relation Verification
Taiheng Pan
Conversational
memory retrieval optimizes relevance, yet a retrieved
memory can be relevant and simultaneously outdated: a later turn updates, corrects, or supersedes it. ConvMemory v3 adds a validity context layer that detects and surfaces this update evidence through target-conditioned relation verification, sitting after the v1/v2
retrieval path. The core mechanism is a dual-evidence gate that conditions a relation judgment on the specific target proposition, scoring a (target, source) pair through the product of a MiniLM slot head and a DeBERTa-v3 slot head and gating it by conservative event/operation evidence. On a synthetic multi-hop validity
benchmark the gate reaches 90.12% +/- 1.73 accuracy; through a real-data feedback loop that mines failure patterns but trains on synthetic pairs only, the verifier transfers to Memora role binding with zero target-side labels, reaching 98.8% +/- 0.9 group-all-correct. The deployed layer preserves
retrieval by default: a context mode attaches structured validity metadata while keeping the candidate set and rank order fixed, and a query-conditioned demote mode is an explicit opt-in for dense current-state workloads, where it raises current-active H@1 from a never-demote baseline of 45.1% to 95.7% +/- 1.2 while protecting non-superseded
memories at 99.4% recall. Six machine-verifiable safety contracts pin the layer's behavior. Multi-hop graph propagation is validated as a mechanism; fully automatic construction of strict prerequisite edges is characterized as a boundary, since strict necessity requires counterfactual world knowledge. This report extends ConvMemory v1 (arXiv:2605.28062) and v2 (arXiv:2606.10842).
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Structure Before Collapse: Transient semantic geometry in next-token prediction
Yize Zhao, Isabel Papadimitriou and Christos Thrampoulidis
Neural Collapse predicts that balanced one-hot classification pushes model representations to be equally far from each other; a symmetric configuration that depends only on the output label and ignores any semantic similarity in the inputs. This creates a puzzle: next-
token prediction
language models are trained predominantly (as context length increases) with one-hot labels: the same context is very unlikely to appear twice in training with different labels. However, they clearly learn latent structural features. That is, despite the one-hot training regime, a
language model's contextual
embeddings represent the fact that the next word in ”Mary broke the ___” is likely to be filled by
tokens in the latent classes of a) medium-sized, b) rigid, c) inanimate nouns. How does gradient descent find such categorical semantic structure when co-occurrence statistics collapse to one-hot sparsity, eliminating any shared next-
tokens among different contexts? To investigate this tension we identify three synthetic controlled settings where inputs have latent semantic factors but are mapped to distinct one-hot labels. We find that semantic geometry emerges early in training, and that representations cluster by shared attributes despite receiving no explicit supervision to do so. This structure is transient: with sufficient capacity and time, the model eventually reaches the predicted symmetric state where all representations are equally separated. We study this phase transition through Gram
matrix analysis and propose a preliminary modification to the commonly used unconstrained features model to capture the emergent semantic geometry.
Read the full abstract ▾