Computation and Language · Cross-listed from cs.AI
Data storytelling meets interpretable machine learning: Decoding AI decisions for non-experts without revealing sensitive data and model details
Lemen Chao, Zixuan Yang, Anran Fang et al. (+2 more)
AI-driven automated decision-making requires both predictive performance and interpretability. Recent advances in interpretable machine learning (IML) provide tools for explaining model predictions, but the technical complexity of these explanations may hinder accessibility to non-experts. To address this challenge, this study integrates data storytelling with IML to enhance the explainability of AI-generated decisions for a broader audience. Following the design science research (DSR) paradigm, this study proposes a formal definition of data storytelling in IML, introduces the DIST Pyramid to align data storytelling with IML, and presents the I-P-O Model to describe their interactions. It further develops an architecture to explain AI decisions through distinct "What-if" and "Why-not" event-generation processes. The architecture also employs data desensitization to protect sensitive input data. To validate the approach, a case study is conducted with the Boston Housing
dataset, using SHapley Additive exPlanations (SHAP) values and large
language models (LLMs) to generate data stories with And-But-Therefore (ABT) structures. An empirical evaluation shows that 76.4% and 74.3% of respondents rated the "What-if" and "Why-not" data stories as more comprehensible, with significantly higher accessibility scores than traditional SHAP visualizations. The paper concludes with the presentation of a narrative interpretation framework that integrates IML and data storytelling, thereby expanding the research scope as well as the practical applicability of AI decision-making.
Read the full abstract ▾
All authors Lemen Chao, Zixuan Yang, Anran Fang, Mingran Sun, Ming Lei
Computation and Language
RESKILL: Explicit Failure Attribution and Structured Repair for Interactive Language Agents
Mengyi Deng, Xin Li, Duyi Pan et al. (+4 more)
Language agents increasingly rely on reusable skills, but post-failure repair is often handled by opaque one-shot reflection: a model generates a skill patch without explicitly maintaining how failure explanations relate to candidate repairs or how unsuccessful retests should influence later edits. We introduce RESKILL, a structured repair framework that maintains an explicit repair state across repair rounds. Given a failed rollout, the framework links failure hypotheses to candidate skill patches, selects local repairs through coverage-based attribution, retests the edited skill set in the environment, and uses retest outcomes to guide subsequent repair updates. The
language model supplies structured repair factors, while the repair procedure records them, compares local skill patches by how well they address active failure explanations, and carries unsuccessful retest outcomes into later repair rounds. We evaluate RESKILL on ALFWorld and TextCraft across three model sizes under fixed repair budgets. RESKILL obtains the strongest final success in all six
benchmark-model settings, improving average final success by 3.7 percentage points over direct repair and 3.3 points over hypothesis-conditioned repair. These results suggest that explicit attribution alone is insufficient; durable improvement emerges when attribution is integrated with repair selection and persistent retest-conditioned update.
Read the full abstract ▾
All authors Mengyi Deng, Xin Li, Duyi Pan, Zilin Wang, Zhiwei Li, Zhijiang Guo, Wei Wang
Computation and Language · Cross-listed from cs.CR
CiteShade: Citation Laundering in Multi-Source Retrieval-Augmented Generation and Its Counterfactual Defense
Guo Fuzheng
Retrieval-augmented generation (RAG) grounds a
language model's answers on retrieved external knowledge and returns each answer with citations that identify its sources. Those citations are the user's audit trail: they let a reader verify a claim without trusting the model. Prior security work on RAG asks whether an attacker can corrupt the answer, leaving the citation channel unexplored. We show that this channel is a new and practical attack surface. We propose CiteShade, the first citation laundering attack to RAG, in which an attacker controlling a single source induces a model to produce an attacker-chosen wrong answer and to attribute it to a trusted source that does not support it, while the evidence for the correct answer remains in context. We formulate the attack as an optimization problem, derive three necessary conditions (
retrieval, generation, and citation) and construct sources satisfying them without any instruction. On multi-source multi-hop question answering the attack raises the wrong-answer rate from 0.01 to 0.68, and source deletion confirms the malicious source is the causal driver in every measured case. Vulnerability tracks a model's propensity to cite rather than its scale, reaching CLR 0.84 under explicit instruction and 0.64 with no instruction at all on the most citation-prone model tested. We then show that perplexity filtering and citation-support checking are each insufficient, and propose a counterfactual defense that verifies which source actually drove the answer.
Read the full abstract ▾
Computation and Language
Empathy Is Steerable but Multi-Axial: Mechanism Geometry and Persona Effects in LLMs
JuHeon Ha, Byounghan Lee, Yunseo Choi et al. (+1 more)
Activation steering has been used to control traits such as honesty, refusal, and sycophancy, yet supportive empathy is evaluated along multiple dimensions that need not correspond to independently controllable activation directions. Using the EPITOME framework, which decomposes supportive empathy into Emotional Reactions, Interpretations, and Explorations, we study three instruction-tuned LLMs and ask whether candidate directions derived from these labels produce distinguishable intervention effects or instead share structure, and how persona
prompts interact with those directions. We find that contrastive activation addition yields a stable middle-layer intervention that consistently shifts the EPITOME proxy scores across models, moving empathy analysis beyond response-level scoring. However, the recovered directions are only partially separable: steering one direction induces off-target shifts, and hand-crafted prompting shifts the empathy profile rather than isolating a single dimension. Persona
prompts substantially change EPITOME scores, but a paired activation-shift decomposition shows that the recovered subspace captures only approximately 3 percent of persona-induced squared activation-shift magnitude at layer 15. Under this EPITOME-based definition, expressed empathy is steerable but multi-axial, and controlling persona-conditioned empathy requires targeting structure beyond individual mechanism directions.
Read the full abstract ▾
All authors JuHeon Ha, Byounghan Lee, Yunseo Choi, Kyung-Ah Sohn
Computation and Language · Cross-listed from cs.CV
Human-Grounded Calibration for Long-Text Image-Text Congruence in Vision-Language Models
Alessandro Gambetti and Qiwei Han
Long-text image–text congruence scoring is increasingly important for
vision-
language systems that must evaluate whether detailed textual descriptions match visual content. However, raw similarity scores from dual-encoder models are difficult to interpret as calibrated congruence measures, especially under the modality gap between image and text
embeddings. This paper proposes Congruency Score (CS), a lightweight calibration layer that maps image–text similarity evidence into a bounded score. Using DOCCI and Urban1k, we evaluate four frozen
vision-
language backbones and show that observed reductions in post-projection centroid distance do not uniformly improve image–text
retrieval performance. Human-grounded evaluations on DOCCI further reveal a trade-off: direct post-hoc calibration preserves high association with human judgments, whereas selected projection-based configurations can reduce threshold-relevant slope and intercept distortions at the cost of
retrieval performance and association strength. These results establish long-text image–text congruence scoring as a calibrated score-estimation problem, where
retrieval performance, human association, and threshold calibration must be evaluated as distinct objectives. CS provides a lightweight way to expose and operationalize this separation.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
IROH: Insightful Ranking Of Humor using Multi-Stage Hybrid Retrieval with Rationale-Distilled LLM Judges for JOKER 2026 Track Task 1 English
Ana-Maria Luisa Mocanu, Sebastian Mocanu, Ciprian-Octavian Truica et al. (+1 more)
Our team, VANGUARD, presents IROH (Insightful Ranking of Humor), a three-stage
retrieval system for JOKER Task 1 English at CLEF 2026, achieving first place on the leaderboard with 0.6347 MAP. Our pipeline combines hybrid sparse-dense
retrieval, cross-encoder reranking, and a LoRA-adapted Large
Language Model judge ensemble. We employ Gemma 4 to generate query-aware rationales under two
prompt strategies, generic and typed, and produce up to four types of structured hard negatives for training data construction. Through an ablation across three cross-encoder architectures, four dense embedders, and eight judge configurations, our key findings are threefold: (1) the rationale-distilled judge is the primary driver of ranking quality, whereas appending rationales to the first-stage index contributes negligibly; (2) structured hard negatives degrade generalisation in nearly all configurations despite inflating local validation scores; and (3) across the components we ablate, the lighter, better-calibrated model is competitive with or stronger than its larger counterpart, with the generic-rationale Qwen2.5-7B judge (0.6055 MAP) outperforming every Gemma-4-31B configuration, and the advantage of generic over typed rationales is concentrated almost entirely in the smaller model.
Read the full abstract ▾
All authors Ana-Maria Luisa Mocanu, Sebastian Mocanu, Ciprian-Octavian Truica, Elena-Simona Apostol
Computation and Language
Through the Eyes of the Beholder: Biometric and Demographic Conditioning for Multimodal Sexism Detection
Ana-Maria Luisa Mocanu, Sebastian Mocanu, Ciprian-Octavian Truică et al. (+1 more)
Detecting sexism on the internet is a fundamentally subjective task; our team, VANGUARD, addresses this challenge in the EXIST 2026 Task 2 by proposing a human-centered
multimodal framework that analyses and incorporates the psychological and demographic characteristics of human annotators into the detection pipeline. We fuse five input modalities through a cross-
attention architecture with Feature-wise Linear Modulation conditioning. Meme text is extracted and visually described with Gemma 4, then augmented by automatic translation between English and Spanish with NLLB-200. Text and image representations are produced by LoRAadapted XLM-RoBERTa and CLIP encoders and fused with sensor features encoded by a pretrained autoencoder. To model annotator subjectivity, we frame Subtask 2.1 as a label distribution learning problem, optimizing a Kullback-Leibler divergence loss over the full annotator label distribution. At inference time, predictions are produced by soft-voting between the deep
multimodal network and a complementary SVM trained on stylometric and physiological features. Our best submission ranks 29th out of 114 on Subtask 2.2 (source intention) under soft evaluation, and the normalized ICM scores remain above the baseline on Subtasks 2.1 and 2.2, indicating that annotator-centered conditioning contributes a usable signal. We release our full pipeline and analysis to support reproducible human-centered modeling.
Read the full abstract ▾
All authors Ana-Maria Luisa Mocanu, Sebastian Mocanu, Ciprian-Octavian Truică, Elena-Simona Apostol
Computation and Language
Can We Trust the Judges? Validation of Factuality Evaluation Methods via Answer Perturbation
Sarra Gharsallah, Adele Robaldo, Mariia Tokareva et al. (+5 more)
Evaluating the factual correctness of large
language models (LLMs) is vital for many applications. But are our evaluation tools themselves trustworthy? Despite the rise of factuality-based metrics, their sensitivity and reliability remain underexplored. This paper introduces a meta-evaluation framework that systematically tests these metrics using controlled corruptions of gold standard answers. Our method generates ranked outputs with known degrees of degradation to probe how metrics capture nuanced changes in truthfulness. Our experiments reveal that pipeline-based methods, such as the RAGAS's factual correctness metric, better track degradation than LLM-as-judge approaches. We also propose a new variant of the factual correctness metric that provides a competitive and cost-efficient.
Read the full abstract ▾
All authors Sarra Gharsallah, Adele Robaldo, Mariia Tokareva, Giovanni Gatti Pinheiro, Ilyana Guendouz, Raphaël Troncy, Paolo Papotti, Pietro Michiardi
Computation and Language
Don't Count the Edits, Judge by the Outcome Alone: Reward-Based Evaluation for Grammatical Error Correction
Hayeong Ryu, Sunhee Jo, Seunguk Yu et al. (+1 more)
Grammatical error correction (GEC) evaluation has traditionally relied on reference or edit overlap, which can penalize valid rewrites that differ from gold corrections. Reference-free metrics reduce this dependence, but evaluating whether a fluent output is a valid correction of the source remains challenging. We propose SURE, a source-conditioned reward evaluator trained on within-source preferences spanning minimal-edit and rewrite-oriented corrections. SURE jointly learns an overall reward with criteria-level supervision for grammaticality, faithfulness, and fluency, together with span-level grounding for source-side error resolution. Experiments on SEEDA show that SURE performs competitively against strong baselines, with particular gains on rewrite-style corrections and more disentangled criteria-level diagnostics. Our code is available at https://github.com/hayeonggg/SURE.
Read the full abstract ▾
All authors Hayeong Ryu, Sunhee Jo, Seunguk Yu, YoungBin Kim
Computation and Language · Cross-listed from cs.AI
Option-Aware Retrieval and Task-Specific VLM Adaptation for Medical VQA
Tristan Kirscher, Niklas C. Koser and Soren Pirk
We describe our submission to the MedReason 2026 challenge, covering multiple-choice (MCQ) and open-ended (OE) medical visual question answering (VQA) under fully offline, containerized inference. Our first finding is that MCQ
retrieval must compare answer semantics rather than answer labels: labels are independently assigned per question, so copying a retrieved neighbor's label transfers no useful information, whereas scoring each current option's text against correct-answer text from similar training cases raises
retrieval-only accuracy from 20.0% to 57.5% on a 200-case
retrieval-excluded development holdout. Our second finding attributes the submitted system's accuracy: holding the task-specific MCQ Low-Rank Adaptation (LoRA) adapter fixed and varying the number \(k\) of in-
prompt retrieved examples changes accuracy by at most one case — 187/200 (93.5%) at both \(k=0\) and the adapter's training-time \(k=1\), 188/200 (94.0%) at the packaged runtime's default \(k=3\) — and the submitted confidence-gated override adds no net accuracy on top of \(k=3\), selecting the VLM in 198/200 cases. With the final MCQ adapter fixed,
retrieval changes accuracy by at most one case, and gating provides no net gain. On 20 OE cases,
token-F1 and RaTEScore~\cite{zhao2024ratescore} decrease as \(k\) grows, but paired sign tests on
token-F1 differences are nonsignificant (\(p \ge 0.29\)); a single-annotator comparison found 6/20 wrong-anchor errors for the final configuration and 14/20 for an earlier configuration that jointly differed in routing, adapter, and prompting. The system reaches 94.0% MCQ accuracy on the development holdout and 93.20% on the organizer's official pre-evaluation, versus 29.43% for the off-the-shelf reference baseline, while both of the organizer's open-ended scores are lower than that baseline's (ground-truth agreement 1.245 versus 1.588, visual accuracy 1.995 versus 2.696, each out of 4).
Read the full abstract ▾
Computation and Language
To Each Language Its Tokenizer: Modular Tokenizers for Efficient Multilingual LLMs
Franck Signe, Hippolyte Pilchen, François Yvon et al. (+1 more)
Multilingual Large
Language Models (LLMs) traditionally rely on a single vocabulary shared by all supported
languages, which can lead to uneven compression across them. Moreover, their large
embedding and output
matrices increase
memory usage and slow inference, notably for small-scale models. It is also wasteful as models are often used for only a subset of
languages. To address these issues, we introduce a modular framework for multilingual model training. First, we propose methods to learn large modular BPE and Unigram tokenizers that enable extraction of subtokenizers tailored to any
language subset. These subtokenizers achieve compression on par with monolingual tokenizers and improve cross-lingual
fairness. Second, we design a pretraining strategy that samples subtokenizers to form batches, restricting predictions to the relevant vocabulary subset and allowing efficient training despite a large vocabulary. This supports efficient inference with any combination of
language-specific vocabularies. Therefore, it reduces
memory usage and speeds up inference in models without sacrificing performance.
Read the full abstract ▾
All authors Franck Signe, Hippolyte Pilchen, François Yvon, Édouard Grave
Computation and Language
Psychosis involves a deficit of information compression in connected speech
Samuele Vallisa, Claudio Palominos, Rui He et al. (+6 more)
Large
language models (LLMs) with human-like performance on linguistic tasks have transformed the study of
language in neurodiverse conditions. LLMs provide representations of linguistic input in the form of high-dimensional vectors (
embeddings), and next-
token predictions computed from these
embeddings. Previous crosslinguistic evidence suggests a complexity reduction in the form of both lower intrinsic dimensionality (ID) of LLM representations and higher mean surprisal (prediction error) in psychosis. We hypothesized that these metrics reflect a general deficit of information compression in psychosis, linked to grammatical organization as what enables predictions in
language.We operationalized surprisal difference as the difference between surprisal as estimated from word frequency and surprisal as based on a contextual LM, which is sensitive to grammatical organization over and above lexical concepts. Using a
dataset of 144 Turkish speakers, including 106 patients with schizophrenia-spectrum disorders (SSD) - 56 with chronic schizophrenia (SZH), 33 with first-episode psychosis (FEP), and 17 with schizoaffective disorder (SZA) - and 38 healthy controls. We report: (1) Surprisal difference is attenuated in all clinical groups relative to controls, independently of word count; (2) Compressibility (intrinsic dimension) is reduced in SZH and FEP; (3) Syntactic complexity and compressibility both predict surprisal difference. These results, further refining an alteration in the geometry of the semantic space in psychosis as previously attested, suggest a broader deficit in information compression in this disorder, with a mechanistic underpinning in the operations of grammar.
Read the full abstract ▾
All authors Samuele Vallisa, Claudio Palominos, Rui He, Emre Bora, Burcu Verim, Cemal Demirlek, Berna Yalincetin, Philipp Homan, Wolfram Hinzen
Computation and Language · Cross-listed from cs.AI
Beyond Safe Answers: Segment-Aware Listwise Alignment for Reasoning Safety in Large Reasoning Models
JungMin Yun, Junehyoung Kwon, Hayeong Ryu et al. (+3 more)
Large
Reasoning Models (LRMs) pose a dual-surface safety challenge: both intermediate
reasoning traces and final answers can contain harmful content. Existing
alignment methods often operate at the whole-response level, allowing unsafe
reasoning to be masked by a safe-looking final answer. We propose Segment-aware Listwise Target DPO (SaLT-DPO), which addresses this gap through three mechanisms: (1) segment-aware listwise
alignment that decomposes responses into
reasoning and answer segments, independently scores each segment's safety, and aligns length-normalized segment rewards with soft target distributions over multiple candidates; (2) joint safety coherence regularization that applies a weakest-link principle to promote safety consistency across both segments; and (3) utility anchoring on benign
prompts to mitigate over-refusal and
reasoning degradation. Experiments on three LRMs show that SaLT-DPO consistently reduces unsafe rates for both
reasoning and answer segments while mitigating degradation in benign compliance and preserving general
reasoning performance. Ablation studies demonstrate the complementary contributions of its components.
Read the full abstract ▾
All authors JungMin Yun, Junehyoung Kwon, Hayeong Ryu, Byeonggeuk Lim, Hoejoon Kwon, YoungBin Kim
Computation and Language
Authorship attribution and aesthetic evaluation of AI poetry: a case study with Haiku
Livia Oddi, Simone Scardapane, Toru Sugimoto et al. (+1 more)
This paper investigates the generation and human evaluation of Japanese haiku by contemporary Large
Language Models (LLMs), focusing on authorship perception and aesthetic judgment within a constrained poetic form. Using a few-shot prompting strategy, Japanese haiku were generated across a heterogeneous set of large
language models, including open- and closed-source systems, medium-scale and large-scale architectures, models with native or adapted Japanese support, and multilingual proprietary models. These AI-generated haiku were combined with human-written ones and presented in a questionnaire distributed to students at Japanese universities in Tokyo. The survey assessed whether respondents could distinguish between AI-generated and human-written haiku and which cues informed their judgments. Recognition accuracy varied across models. GPT-5, Gemini 2.5, and StableLM-7B performed at approximately chance level (approx 0.50), whereas LLM-JP, Gemma-2B, and LLaMA-2 showed moderate detectability (approx 0.59-0.67). However, recognition was strongly item-dependent. Ratings of fluency, coherence, poeticness, and related aesthetic dimensions predicted perceived humanness but not correct classification, indicating an attribution bias linked to aesthetic evaluation and revealing a dissociation between aesthetic evaluation and true authorship detection. The extended analysis additionally examines generation-constraint adherence, participant-level characteristics, and exploratory LLM-based evaluations of haiku authorship. Overall, the findings suggest that as LLMs improve, surface-level creative plausibility may reduce reliable human discrimination within constrained poetic settings.
Read the full abstract ▾
All authors Livia Oddi, Simone Scardapane, Toru Sugimoto, Donatella Genovese
Computation and Language
Evaluating Losslessness in Speculative Decoding Under Finite-Precision Inference
Ilya Koziev, Leonid Sinev and Ivan Oseledets
Lossless speculative decoding is typically defined at the algorithmic level: a speculative procedure proposes multiple
tokens and a verification procedure is designed to preserve the output trajectory of an autoregressive reference model exactly. In practical neural inference, however, this guarantee is implemented using finite-precision floating-point computations, and discrete
token selection can amplify small numerical differences into divergent generation trajectories. We investigate this distinction using Orthrus, a hybrid autoregressive-
diffusion architecture that performs self-drafting and self-verification within a frozen autoregressive backbone, as a representative case study. Across 1,190
prompts from 12 domains, exact trajectory matching under BF16 occurs for only 45% of the authors' checkpoint generations and 43% of those from our independently trained model. The probability of matching is strongly associated with the response-conditional perplexity of the autoregressive reference, indicating that trajectory divergence is not uniform across inputs. Despite these divergences, Orthrus does not exhibit systematic degradation on the evaluated downstream tasks. In contrast, FP32 inference yields exact trajectory matching on all evaluated
prompts. These results demonstrate a gap between algorithmic losslessness and its implementation under finite-precision arithmetic, and motivate evaluating lossless speculative decoding at the level of exact generation trajectories as well as downstream task performance.
Read the full abstract ▾
Computation and Language
Temperature Fragility and the Conditional Benefits of Truncation Sampling
Francesco La Rosa
Large
language models generate text by sampling each
token from a predicted distribution, and a temperature parameter sets how far the draw strays from the most probable
tokens. Truncation samplers such as top-p and min-p discard the least probable
tokens before the draw, so that sampling at high temperature stays coherent. Their reported accuracy gains come from temperatures of 1.5 to 3, while the defaults of deployed systems cluster between 0.6 and 1.0. Whether they change accuracy at those defaults, and for which models, has not been measured. We test thirteen open-weight models on GSM8K and MMLU-Pro at temperatures 0.7, 1.0, and 1.3 in one controlled pipeline, ten of them under eight decoding configurations. Six of the thirteen models lose 17 to 38 accuracy points on MMLU-Pro between 0.7 and 1.3, and the other seven lose at most 10. The lost accuracy comes from generations that run to the
token limit or never state an answer. These results suggest that truncation samplers improve accuracy primarily when higher temperatures substantially degrade model performance. Where accuracy remains stable across temperatures, none of the tested truncation samplers improves on plain temperature sampling.
Read the full abstract ▾
Computation and Language
Turkish MMLU Pro: Traceable Option Augmentation and Its Validity Limits in Turkish Multiple-Choice Evaluation
M. Ali Bayram
Adding answer options can lower multiple-choice scores without improving assessment validity. Turkish MMLU Pro examines this distinction using 12,000 Turkish-source questions across 58 sections. Each question retains its stem, five original options and source key, and receives five options copied from other questions in the same section. Sentence-
embedding retrieval proposes candidates; a
language model selects existing identifiers. Deterministic verification reconstructs all 60,000 additions. A 25-model calibration exposes scoring and generation-budget effects. Five evaluations produce source-key accuracies of 34.8%-81.4%. On 981 shared questions, one API-served model falls from 93.7% with five choices to 83.1% with ten; 102 of 115 lost correct responses select borrowed options. The decrease is 24.4 percentage points on heuristically flagged negative stems and 5.9 points elsewhere. A completed human-checked audit of 200 sampled questions, with undocumented reviewer tool use, yields 47 and 31 multiple-answer judgments across the two record sets, 25 of the latter unresolved. These records support concern about ambiguity, while their dependence and incomplete reviewer-method documentation limit validation. Because order and labels also change, the paired comparison measures augmentation as implemented. The contribution is a traceable construction and an analysis of its validity limits, not evidence that lower ten-choice scores measure knowledge better.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
MarKey: Marginal Utility Guided Greedy Keyframe Selection for Long Video Understanding
Hongchang Shi, Jinpeng Hu, Ao Wang et al. (+4 more)
Long-
video understanding remains challenging for
multimodal large
language models (MLLMs) because densely encoding long frame sequences is computationally expensive, while uniform sampling under a limited visual budget can miss sparse yet decisive evidence. Recent training-free keyframe selection methods have enabled more efficient inference and yielded promising performance gains. However, many existing methods score frames largely in isolation without explicitly considering how each candidate complements the currently selected subset, potentially resulting in redundant selections and incomplete evidence coverage. To address this limitation, we propose MarKey, a training-free framework that formulates keyframe selection as subset-aware greedy optimization. At each iteration, MarKey scores each candidate using a tractable surrogate that jointly accounts for query relevance, marginal coverage gain, and context-dependent redundancy, and selects the frame with the highest utility. To make this iterative subset-aware evaluation efficient, MarKey uses a compact set of representative anchors to approximate full-
video coverage and a bounded window of previously selected frames to limit context-dependent comparisons. Experiments on six
benchmarks spanning holistic
video understanding, human-centric
video understanding, and open-ended
video understanding demonstrate that MarKey consistently outperforms existing methods. Further analyses show robust gains across different MLLM backbones, model scales, and frame budgets.
Read the full abstract ▾
All authors Hongchang Shi, Jinpeng Hu, Ao Wang, Wenzheng Zhou, Hui Ma, Feng Li, Zenglin Shi
Computation and Language
SlopShape: Identifying AI-Generated Commercial Web Content
Jochen Madler
Word-level detectors identify unedited AI-generated text almost perfectly, but the literature documents their brittleness under rewording, and a word-level score neither characterizes a text nor identifies which AI model wrote it. We ask whether AI-generated text can be identified one level deeper, from structural signatures: how information is presented, in what order, with what evidence, and in what voice. We replicate StoryScope (Russell et al., 2026), which showed such patterns for AI-generated fiction, on commercial content: 2,250 pre-ChatGPT human blog posts from 268 company domains against 11,250 AI mirrors from five frontier models. A 203-feature instrument, applied by an LLM and validated in a human gold-annotation session (human-human kappa 0.939, human-model 0.951), detects AI posts from its 176 structural features alone at 97.0 macro-F1 on held-out companies, nearly unchanged (96.1) when every AI post is reworded by its own model. The signal characterizes and attributes: AI posts share a tidy, self-announcing shape, 68.6% are attributed to the correct source against a 16.7% chance rate, and human posts occupy rare structural configurations. All effects replicate StoryScope's, consistent in direction and at least as large in magnitude. We release pipeline, instrument,
prompts, code, and aggregate artifacts.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments
Sibo Zhu, Shicheng Fan, Xinyue Wang et al. (+3 more)
Digital
agents must often adapt to new environments whose interfaces, tools, and failure modes are not fully captured by pretrained models. We introduce RSIAgent, a training-free multi-
agent framework for recursive self-improvement through autonomous
memory construction. RSIAgent coordinates curriculum, actor, and verifier
agents to continually explore the environment, validate outcomes, and retain environment-specific knowledge, including reusable causal relationships between actions, conditions, and consequences. It further adopts a broad-then-deep exploration strategy, combining parallel broad recursive self-exploration for discovering diverse environment structures with focused deep self-exploration for uncovering hard cases, hidden constraints, boundary conditions, and previously unknown causal dependencies. The resulting
memory is frozen and can be directly reused for downstream tasks without updating model parameters. Experiments on OSWorld-v2 and
Agent's Last Exam show that RSIAgent substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.
Read the full abstract ▾
All authors Sibo Zhu, Shicheng Fan, Xinyue Wang, Wenyi Wu, Kun Zhou, Biwei Huang
Computation and Language · Cross-listed from cs.AI
Parameter-Efficient Adaptation of Pretrained Language Models for Time-Series Forecasting
Tamanna Kumavat, Georg Brunner and Kyriakos Flouris
We study the adaptation of pretrained
language models to univariate time-series forecasting through a parameter-efficient transfer learning framework, with the goal of understanding which design choices drive effective cross-modal transfer. While
language models operate on discrete textual
tokens, time series consist of continuous numerical observations with temporal dependencies. To bridge this modality gap, we project fixed-length time-series patches directly into the
embedding space of a pretrained GPT-2 backbone, bypassing textual tokenization and treating the
Transformer as a generic sequence encoder. Through controlled ablation studies on seven
benchmark datasets spanning energy, weather, traffic, and finance, we analyze the effects of (i)~representation strategy (continuous
embeddings versus textual serialisation), (ii)~adaptation regime (frozen backbone versus partial or full fine-tuning), (iii)~architectural components such as adapters, pooling strategies, and prediction heads, and (iv)~input context length. Continuous patch-based
embeddings consistently outperform textual prompting and randomly initialised backbones. The adapted pipeline attains MASE within the range of specialised forecasting architectures while updating less than 1% of total model parameters. Results further indicate that freezing the pretrained backbone and training lightweight projection and adapter modules provides a favourable accuracy–efficiency trade-off with stable behaviour across varying context lengths.
Read the full abstract ▾
Computation and Language
Dynamic Semantic Compression for Efficient Latent-Space Inference in Large Language Models
Peipei Li, Dongsen Zhang, Yuchen Liu et al. (+1 more)
Large
Language Models (LLMs) primarily perform inference at the
token level, resulting in substantial
memory overhead and compromised computational efficiency. In this paper, we propose a Dynamic Semantic Extraction and Inference (DSEI) framework, which achieves segment-level inference within the latent space through a two-stage training strategy. First, we construct a Dynamic Semantic Autoencoder (DSAE) via self-supervised learning. DSAE dynamically extracts segment-level semantics and compresses them into compact latent representations via adaptive semantic weighting and gated fusion. Subsequently, we integrate the DSAE into the LLM architecture and train the model to infer over dense latent space. DSEI substantially reduces both input and generation sequences and significantly enhances inference efficiency. Extensive experiments conducted on the Wanjuan
dataset demonstrate that DSEI reduces perplexity by 48% compared to static sentence-level latent inference baseline. Furthermore, compared to standard LLMs using
token-level inference, DSEI accelerates inference speed by 2.5$\times$ and reduces
memory overhead by 90%.
Read the full abstract ▾
All authors Peipei Li, Dongsen Zhang, Yuchen Liu, Wenjun Xu
Computation and Language · Cross-listed from cs.IR
Clean Scores, Buried Evidence, and Confident Wrong: A Receipt-Based Audit of Frontier Agentic QA
Luis M. Sánchez
Frontier models score well on shallow document/chart reading tasks. In a controlled data-room audit, moving evidence into buried conditions reduced accuracy, increased forced declarations, increased tool calls, and increased cost per correct answer. Confidence and
benchmark calibration did not fully capture wrong answers; a documented production incident shows fabricated structural claims can be mixed with accurate numeric tables. Agentic evaluations need claim-level receipts (statement-level provenance, not answer-level scores), condition-aware scoring, and human-
adversarial verification - an auditing discipline, not a leaderboard. The setting we measure is financial due diligence; the setting we are building toward next is defense staff work, where the same buried-evidence shape appears. In both, the model is not a party to the consequences; the person who signs is. In plain terms: in the documented cases we examine,
agents can pair accurate numbers with confident fabricated explanations, and the burden of proof must therefore move from the model to the evidence trail.
Read the full abstract ▾
Computation and Language · Cross-listed from eess.AS
Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation
Daxin Tan, Dehua Tao, Chengxi Deng et al. (+2 more)
Autoregressive generation of interleaved text and acoustic
tokens is a common approach to spoken-response generation in
speech large
language models. Although this design enables streaming generation with explicit textual guidance, generated acoustic
tokens become part of the context for subsequent text predictions. Given identical
speech inputs, we observe markedly lower answer accuracy for the internal text generated in
speech-to-text-and-
speech (S2TS) mode than for
speech-to-text (S2T) responses. We term this discrepancy the output-mode gap (OMG). To reduce OMG, we propose Joint-Output On-Policy
Distillation (JO-OPD), which distills the model's stronger S2T policy into joint generation using student-generated S2TS trajectories. At each text position, the S2T teacher provides soft targets from a text-only projection of the student's preceding outputs, while the student predicts from the corresponding full interleaved history. A preservation objective further regularizes native non-text predictions. Experiments on Step-Audio-2-mini and Baichuan-Audio-Instruct reveal OMG across two interleaved generation architectures. On Step-Audio-2-mini, JO-OPD reduces OMG from 42.87 to 16.26 percentage points on Spoken-MQA and from 29.72 to 13.04 points on
speech-rendered GSM8K, with little change in S2T accuracy and substantially larger reductions than matched SFT baselines. ASR-based evaluation further shows a 7.49-point improvement in spoken-answer accuracy on Spoken-MQA.
Read the full abstract ▾
All authors Daxin Tan, Dehua Tao, Chengxi Deng, Hanlin Zhang, Xiao Chen
Computation and Language
When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis
Kaiyuan Liu, Qiuyang Mang, Bo Peng et al. (+6 more)
Large
language model (LLM)
agents allocate test-time compute adaptively as they revise solutions, use tools, explore alternatives, and decide when to stop. This test-time strategy makes it difficult to measure how
agent performance scales. We study open-ended tasks that provide continuous scores for intermediate submissions, making progress observable throughout long trajectories. We propose Elo-per-
token analysis, which tracks the best solution found at each
token budget and uses a Bradley-Terry model to aggregate within-task orderings into Elo ratings across tasks with different score scales. We apply it to four general-purpose
agents on four open-ended
benchmarks, with sessions of up to 100M
tokens, and to three feedback-driven LLM optimization harnesses in controlled single-task interventions. Independent sampling provides a theoretically characterized reference, for which Elo grows linearly with log compute. Against this reference,
agents can initially convert
tokens into Elo faster than independent sampling, but their marginal gains diminish and eventually fall below the reference. In contrast, the strongest historical human contestants improve superlinearly over contest time on shared AtCoder Heuristic Contest tasks, providing evidence of continual learning and substantial headroom after
agents slow down. We define the scaling inflection point as the per-session budget where marginal Elo gains match the independent-sampling reference. Using this point as the per-session budget, we split 100M
tokens across parallel sessions on FrontierCS Polyomino Packing, gaining +264 Elo over one long session and +355 over ten short sessions.
Read the full abstract ▾
All authors Kaiyuan Liu, Qiuyang Mang, Bo Peng, Wenhao Chai, Hanchen Li, Shreyas Pimpalgaonkar, Luke Zettlemoyer, Alex Dimakis, Alvin Cheung
Computation and Language · Cross-listed from cs.AI
Mingzhou Jiang, Peixi Wu, Hang Cheng et al. (+7 more)
Universal
multimodal embedding (UME) maps
multimodal inputs into a shared
embedding space for diverse
retrieval tasks. Recent methods improve
embeddings through Chain-of-Thought (CoT)
reasoning optimized with GRPO using
retrieval rewards. However, existing methods overlook the mismatch bettween candidate-aware
retrieval supervision and input-only CoT generation: (1)trajectory-level rewards convey
retrieval outcomes without explicitly identifying the input-supported evidence that distinguishes the positive from hard negatives; (2) input-only generation cannot directly assess whether further
reasoning improves
retrieval, potentially producing redundant CoTs with substantial latency. To bridge this gap, we propose Reason What Matters (ReWAM), a
retrieval-grounded framework that aligns candidate-aware supervision with input-only generation. Specifically, we introduce
Retrieval-Aware Self-
Distillation (RASD), which extracts privileged guidance from input-supported facts and evidence distinguishing the positive from hard negatives. Conditioned on this guidance, an on-policy self-teacher provides
token-level feedback to refine credit assignment, directing policy updates toward
retrieval-relevant
reasoning grounded in the input. We further propose
Retrieval-Adaptive Inference (RAI), which learns a
retrieval-aware stopping criterion from prefix-level
retrieval feedback. It stops redundant
reasoning without candidate access and uses speculative decoding to further reduce CoT latency. Extensive experiments on MMEB-V2 and MRMR demonstrate that ReWAM achieves state-of-the-art
retrieval performance while delivering up to 5x the inference throughput of competitive explicit-CoT UME methods. ReWAM thus enables high-quality
retrieval through efficient input-only
reasoning, making explicit CoT practical for corpus-scale
multimodal retrieval. The code will be publicly available.
Read the full abstract ▾
All authors Mingzhou Jiang, Peixi Wu, Hang Cheng, Yunhao Zhou, Biao Yang, Wei Yuan, Yun Li, Fan Yang, Wenwu Ou, Honghui He
Computation and Language
Artificial entrepreneurial cognition: Locating and causally steering an opportunity recognition dial inside large language models (LLMs)
Christian Fisch, Angela Altmeier, Martin Obschonka et al. (+2 more)
Entrepreneurial cognition is a foundation of entrepreneurship research. Yet the growing involvement of large
language models (LLMs) in entrepreneurial work extends the cognition question beyond human actors to systems whose internal representations remain largely unexplored. We introduce artificial entrepreneurial cognition, the functional organisation of entrepreneurship-relevant representations and computations inside artificial intelligence (AI) systems. We bring mechanistic interpretability into entrepreneurship research through representation engineering. Focusing on opportunity recognition (OR), we construct 636 matched OR-present and OR-absent scenario pairs and recover an OR direction in Llama 3.1 8B-Instruct. Rather than infer the construct from outputs, we intervene directly on this direction, steering the model up and down along what we call the opportunity recognition dial, and its opportunity judgments shift with it. To our knowledge, this is the first causal intervention on an internal representation of an entrepreneurship construct inside an LLM. Held-out tests, lexical and topical controls, behavioural ablation, and
geometric comparisons show that the direction is recoverable, consequential, and distinct from the opportunity evaluation and exploitation directions, although steering it also shifts judgments about these neighbouring stages. Recovery, signed steering, and
geometric separation hold across four additional LLMs spanning different scales and families. These results give the contested distinction between opportunity recognition and evaluation a concrete representational form inside AI systems. More broadly, they establish internal representations as a new object of entrepreneurship inquiry and show how entrepreneurship theory can guide their identification, causal manipulation, and interpretation.
Read the full abstract ▾
All authors Christian Fisch, Angela Altmeier, Martin Obschonka, Michal Kosinski, Pin Ni
Computation and Language
Semiotic Relations and Proof Methods: A Cross-Genre Study of Argument Structure with Large Language Models
Edirlei Soares de Lima, Marco A. Casanova and Antonio L. Furtado
When a direct proof of a statement $S$ seems hard or even impossible to obtain, there may exist another statement (or set of statements) $S^{*}$, somehow related to $S$, on the basis of which $S$ can be proved. In order to investigate what options can be used to move from $S$ to $S^{*}$, four kinds of semiotic relations inspired by the four master tropes of semiotic research are briefly reviewed. Specifically, our syntagmatic, paradigmatic, antithetic and meronymic relations correspond, respectively, to metonymy, metaphor, irony and synecdoche. It is suggested that these four semiotic relations determine the options to move from $S$ to $S^{*}$, leading to proof by inference, proof by analogy, proof by contradiction, and proof by case analysis. To examine how the four relations are actually used across different kinds of argument, we complement the framework with an empirical study. We turn the four relations into explicit operational definitions and apply them to a cross-genre corpus of mathematical, legal, and everyday argument using a panel of large
language models. We find that the relations are used very unevenly across genres: mathematical proofs draw on all four, whereas legal and everyday
reasoning rely almost entirely on inference.
Read the full abstract ▾
Computation and Language
What Limits Us? Analyzing Self-Reported Limitations in NLP Research
Tawan Thaepprasit, Peeranuth Kehasukcharoen, Ding Wang et al. (+3 more)
Since late 2022, a Limitations section has become mandatory at many top-tier NLP conferences. The growing number of accepted papers at these venues has resulted in a vast corpus of self-reported limitations that cannot all be manually reviewed, yet remains systematically unanalyzed. Therefore, in this paper, we conduct a large-scale analysis of the Limitations sections from ACL and EMNLP papers published between 2020 and 2025 to understand what researchers disclose about their own work. To do so, we implement a novel human-AI framework for iterative hybrid qualitative coding. This framework enables us to investigate trends in self-reported limitations over time, their correlations with specific paper attributes, and the writing patterns that recur around these disclosures. Our findings offer a critical reflection on the diverse reported challenges as well as the self-reporting practices of researchers in the NLP community.
Read the full abstract ▾
All authors Tawan Thaepprasit, Peeranuth Kehasukcharoen, Ding Wang, Remi Denton, Peerapon Vateekul, Piyawat Lertvittayakumjorn
Computation and Language
MUSE: A Theory-Harnessed Story Engine for Vibe Narrativizing
Jianxiang Ma, Xiaocui Yang, Daling Wang et al. (+5 more)
LLMs have been able to generate fluent prose, but high-quality stories also require coordinated decisions about plot, character, and
language across planning, drafting, and revision. We formulate Vibe Narrativizing as turning natural-
language writing requirements into a finished story. MUSE, a Theory-Harnessed Story Engine, addresses two bottlenecks: rule quality and sustained rule realization. Story theory supplies the rules, and a practical
agent harness puts them to work. Knowledge engineering organizes Robert McKee's theory through rule atomization, semantic consolidation, mechanism abstraction, a single source of truth, and layered disclosure; typical examples clarify judgments that depend on context and aesthetic purpose. The harness preserves story decisions in intermediate deliverables across design, character performance, scene composition, and revision. Context engineering supplies each role with relevant guidance and decisions; a masterwork corpus provides inspiration and prose references. A worked example follows a requested object from its thematic role to climactic actions. Across four base models, MUSE improves WritingBench by 1.1 to 6.2 points over zero-shot generation; it is the only multi-stage system in our comparison to do so. It also raises LongStoryEval by more than ten points on three of the four models. ConStory-Bench consistency error density remains in the low single digits for all four models, below every reproduced story-system baseline on three of the four models. Ablations locate the largest quality contribution in structural design, voice-specific effects in the character path, and further gains in revision.
Read the full abstract ▾
All authors Jianxiang Ma, Xiaocui Yang, Daling Wang, Mingfu Zhang, Zijing Wang, Yuesong Hou, Yichen Gao, Junzhao Huang