Computation and Language
En-ViMedNER: An English-Vietnamese Parallel Biomedical Corpus with UMLS Semantic Type Annotations
Nhu Vo, Phuong Nguyen, Nu Uyen Phuong Le et al. (+4 more)
Biomedical Named Entity Recognition (NER) is fundamental to healthcare AI applications, including clinical decision support and medical information extraction. While corpora with Unified Medical
Language System (UMLS) annotations, such as MedMentions, have driven progress in English biomedical NER, no comparable resource exists for Vietnamese. This paper presents En-ViMedNER, the first English-Vietnamese parallel biomedical NER corpus annotated with UMLS semantic types, which are
language-neutral codes providing a shared cross-lingual label space and ensuring direct comparability with existing UMLS-based resources. The corpus contains 4,392 PubMed abstract pairs, 44,892 English-Vietnamese sentence pairs, and 202,949 aligned entity-mention pairs across 21 semantic types adapted from the MedMentions ST21pv
dataset. To balance quality and scalability, we have constructed the corpus through automatic translation, expert post-editing, LLM-assisted label projection, and human verification and adjudication. We characterize En-ViMedNER as a large-scale silver-standard corpus with a human-audited and consensus-corrected mini-test subset. We evaluate En-ViMedNER in two settings: (i) Vietnamese-input/Vietnamese-output biomedical NER and (ii) English-input/Vietnamese-output cross-lingual NER. For Vietnamese NER, we
benchmark Vietnamese-supervised encoder models, English-supervised
multilingual encoder models, and
prompt-based LLMs. The best model achieves an F1 score of 52.70 on the test set and 53.78 on the mini-test set. For cross-lingual NER, we
benchmark encoder-decoder models and
prompt-based LLMs. The best model achieves an F1 score of 45.44 on the mini-test set. We publicly release our corpus, corpus construction pipeline, and baseline models to facilitate future Vietnamese biomedical NLP research.
Read the full abstract ▾
All authors Nhu Vo, Phuong Nguyen, Nu Uyen Phuong Le, Inigo Jauregi Unanue, Dung D. Le, Massimo Piccardi, Wray Buntine
Computation and Language
VibeJam: A User Study Platform for Web Development with Agents
Nishant Balepur, Connor Baumler, Valerie Chen et al. (+3 more)
Programming with AI is increasingly agentic, users
prompt LLMs to directly edit their code and review the changes, with adoption growing especially for web development tasks. Despite this growth, most NLP work uses offline evaluation and lacks support for online studies, losing insights into how programmers truly use coding
agents. We release VibeJam, a browser-based user study platform for users to collaborate with AI
agents to develop websites. VibeJam enables
agent customization and uses the open-source Aider
agent by default, and to mirror downstream use, we add diff review, chat and plan modes, and live website previews. In a pilot study with 55 released, game-based website creation tasks, five experienced AI programmers rate our system as fun, simple, and resembling commercial tools, while 13 junior students use VibeJam to make websites of higher quality than
agents in the same task. We open-source VibeJam to spur extensions and support studies on how coding
agents can help users.
Read the full abstract ▾
All authors Nishant Balepur, Connor Baumler, Valerie Chen, Eunsol Choi, Rachel Rudinger, Jordan Boyd-Graber
Computation and Language
Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation
Nishant Balepur, Paiheng Xu, Wei Ai et al. (+3 more)
Multiple-choice question answering (MCQA)
benchmarks in NLP use number-right scoring (accuracy), but in educational testing, the scoring scheme, the combination of the response mode models follow and the rule for grading responses, is a key design choice that dictates which abilities to reward. We examine how alternatives to number right change what MCQA measures with six education-inspired schemes that assess abilities beyond accuracy: distractor elimination, abstention, confidence calibration, and self-correction. On LLM
benchmarks, these schemes: 1) shift rankings of 31 LLMs beyond rephrased number right
prompts; 2) better predict the LLMs users prefer in LLM Arena; and 3) reveal distinct model capabilities, like that GPT-5 rarely abstains and readily self-corrects, while weaker open-weight models often abstain and hesitate to eliminate choices. Given the benefits of alternative scoring schemes, we discuss ways to extend them to tasks beyond MCQA.
Read the full abstract ▾
All authors Nishant Balepur, Paiheng Xu, Wei Ai, Eunsol Choi, Rachel Rudinger, Jordan Boyd-Graber
Computation and Language
Improving Argument Saliency Coverage in Small LLMs for Long Legal Opinion Summarization via Sequence-Level Distillation
Mohamed Elaraby, Ahmed Elhady and Diane Litman
We show that sequence-level
distillation from a capable long-context teacher model is a simple, annotation-free, and data-efficient strategy for improving argument saliency coverage in long legal opinion summarization, where small LLMs often struggle to retain the most salient argumentative content. Across student model sizes,
distillation consistently surpasses tuning on expert-written summaries in our legal-opinion setting. We further demonstrate that most gains are achieved with as few as ~10 training summaries, highlighting the strong data efficiency of teacher-generated supervision. Finally, we find that summary
distillation is sufficient for improvements:
reasoning-chain
distillation remains competitive with summary-only
distillation, but provides marginal benefit when combined with summary supervision.
Read the full abstract ▾
Computation and Language
ManGo: Manga Active Narrative Grounding Optimization
Hao Qiu, Junyan Wang, Zheyuan Liu et al. (+4 more)
Manga visual question answering requires models to answer questions over panel-based visual narratives, where relevant evidence is distributed across ordered panels, embedded text, recurring characters, and implicit event transitions. This structure makes passive page encoding insufficient, as the model must identify which panels to inspect, what clues to retain, and when the accumulated evidence is sufficient for answering. We propose ManGo (Manga Active Narrative Grounding Optimization), an unsupervised framework for active manga visual question answering. ManGo introduces Active Narrative Sketching (ANS), which iteratively selects panels, extracts concise grounded clues, and decides when to stop, forming a compact question-directed evidence sketch before answer generation. To optimize this behavior without human-annotated answers or rationale paths, ManGo samples multiple ANS rollouts and applies group-relative training with two rewards: answer preference from listwise self-ranking and path consistency from stable ordered panel trajectories. The combined reward is optimized with group-relative policy training, encouraging the model to improve both final answers and the panel-level evidence paths that support them. Experiments on standard manga understanding
benchmarks show that ManGo achieves state-of-the-art performance across different settings.
Read the full abstract ▾
All authors Hao Qiu, Junyan Wang, Zheyuan Liu, Lei Fan, Hong Jia, Lianbo Guo, Zhulin Tao
Computation and Language
GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluation
Yifan Chen, Haitao Li, Qingyao Ai et al. (+4 more)
Large
language models are increasingly used as scalable evaluators for open-ended tasks. However, many LLM judges derive query-specific criteria during scoring, leaving the evaluation requirements insufficiently specified and their coverage difficult to audit. Query-specific rubrics make these requirements explicit, but expert-written rubrics are costly to construct, while existing automatic methods typically rely on inference-time refinement or external supervision. We introduce GenRubric, a self-evolving framework that improves rubric generation from unlabeled queries without requiring additional human annotations during self-
evolution. Our approach is based on rubric-induced self-consistency: independently sampled rubrics for the same query provide partial views of its latent evaluation requirements, and a comprehensive rubric should induce a response that generalizes across these complementary evaluation views. We implement this principle through reinforcement learning, combining a cross-rubric comprehensiveness signal with group-level and criterion-level rewards for rubric quality. We train GenRubric models at 4B, 8B, and 14B scales across multiple domains. Experiments on human-annotated rubric
benchmarks show that self-
evolution improves the agreement between evaluations induced by generated rubrics and those induced by expert-written rubrics. The improvements further generalize to held-out domains, demonstrating the potential of self-evolving rubric generation for scalable and query-specific LLM evaluation. Code and models are publicly available at https://github.com/foggpoy/GenRubric.
Read the full abstract ▾
All authors Yifan Chen, Haitao Li, Qingyao Ai, Fengbin Zhu, Tat-Seng Chua, Min Zhang, Yiqun Liu
Computation and Language · Cross-listed from cs.CV
ContextBias: Controlled Evaluation of Bias Persistence Under Context Shift in Text-to-Image Models
Shaghayegh Kolli, Sina Emami, Moreno D'Incà et al. (+4 more)
Text-to-image models learn associations between concepts - in the case of this paper, people's professions, which we refer to as roles - and visual attributes. These associations can underpin many observed forms of stereotypical bias. A key open question in this area is whether these associations are stable or change when visual representations of people in professional roles are placed in different prompted contexts. We introduce ContextBias, a controlled evaluation framework, and ContextBench, a
benchmark spanning 92 roles and 1,656 semantically controlled
prompts, designed to isolate the effect of contextual variation on role-linked visual representations. Evaluating four state-of-the-art models on 66,240 generated images, we find that placing a role in a semantically unrelated context does not suppress role-linked attributes; instead, cross-role attribute concentration increases (pooled BI $+0.047$). Demographic cues, characteristic garments, and role-specific tools remain highly prevalent across context-free, related, and unrelated conditions, and are robust to semantic
prompt reformulation. Scene composition and camera framing show the greatest context-sensitivity. These findings reveal a form of stereotypical persistence that remains largely invisible to context-free evaluations, highlighting the need for controlled contextual variation in bias benchmarking. Code and
dataset: https://huggingface.co/
datasets/shaghayegh/ContextBias , https://github.com/Sina-Emami/ContextBias
Read the full abstract ▾
All authors Shaghayegh Kolli, Sina Emami, Moreno D'Incà, Pouyan Nejadi, Nicu Sebe, Massimiliano Mancini, Jana Diesner
Computation and Language
Influence-Directed Distillation: Solving the Diversity Bottleneck in Sampled-Token On-Policy Distillation
Run Yang, Runpeng Dai, Jie Sun et al. (+5 more)
Sampled-
token on-policy
distillation (OPD) efficiently transfers capabilities from teacher to student using student-generated
tokens, requiring teacher probabilities only for sampled
tokens. Yet it frequently suffers from diversity
distillation failure: the student's pass@1 improves while its pass@$k$ plateaus, failing to inherit the teacher's diversity. To explain this, we introduce First-Order Local
Entropy Influence, a signed first-order proxy that decouples each update's
entropy effect into the teacher–student log-probability gap and the student's local probability structure, and empirically links
entropy contraction to negative-influence positions. Motivated by this, we propose Influence-Directed Adaptive On-Policy
Distillation (IDA-OPD): rather than relying on costly full-vocabulary Forward-KL objectives, it preserves
entropy-expanding updates while replacing
entropy-contracting ones with divergence-adaptive advantage shrinkage, using only the teacher's sampled-
token log-probability. Experiments on
reasoning-oriented
distillation show IDA-OPD consistently improves pass@$k$, inheriting the teacher's diversity through
distillation, matches the strongest teacher-informed methods at strictly lower cost, and broadly maintains vanilla OPD's pass@1, all without full-vocabulary teacher information.
Read the full abstract ▾
All authors Run Yang, Runpeng Dai, Jie Sun, Jielei Zhang, Fan Zhou, Hongtu Zhu, Peiyi Li, Longwen Gao
Computation and Language
SkillForge: Compositional Skill Synthesis with Verification-in-the-Loop for Generating Formally Verified Dafny Programs
Yanming Liu, Xinyue Peng, Jiannan Cao et al. (+2 more)
Generating formally verified programs from natural
language remains challenging: existing approaches either produce code in a single pass without recourse when verification fails, or rely on open-ended agentic
reasoning that is non-deterministic and opaque. We introduce SKILLFORGE, a framework that decomposes formal code synthesis into a library of atomic, reusable skills, each targeting a specific subtask such as specification inference, body synthesis, invariant generation, error diagnosis, or targeted repair, and defined by a
prompt template, tool binding, and decidable success criterion. A verification-driven harness orchestrates these skills: it submits candidates to the Dafny verifier, diagnoses failures into structured categories, deterministically routes to the appropriate repair skill, and iterates until formal correctness is proved or a budget is exhausted. On a curated
benchmark of natural
language to Dafny specification pairs, SKILLFORGE substantially outperforms both state-of-the-art agentic approaches (including ReAct-style
agents, MCTS-based repair, and RL-guided verification) and traditional iterative baselines, while requiring fewer
tokens and lower latency. Ablation studies confirm that every skill contributes measurably, and the harness converges rapidly with the majority of programs verified on the first attempt.
Read the full abstract ▾
All authors Yanming Liu, Xinyue Peng, Jiannan Cao, Xinyi Wang, Jinbo Su
Computation and Language
EVAR: Evidence-Validated Hypothesis Admission for Budget-Aware Narrative Reasoning
Peilin Liu, Zhiquan Ji and Jinglong Ping
Large
language models (LLMs) often produce fluent but weakly grounded conclusions when
reasoning over non-interactive, long-form narratives. A central failure mode is that unsupported intermediate hypotheses can enter the
reasoning trajectory and contaminate subsequent inference, especially when evidence is scattered across distant parts of the story. To address this problem, we propose EVAR, an evidence-validated hypothesis admission framework for budget-aware narrative
reasoning. EVAR first compiles the narrative into an immutable evidence store of source-linked atomic claims and assigns an instance-specific inference budget from unresolved gaps and uncertainty signals. During refinement, EVAR directly proposes candidate hypotheses for unresolved gaps, constructs hypothesis-conditioned validation challenges, and verifies each candidate against the locked store before admission: supported hypotheses enter the answer-supporting state, unverifiable ones are quarantined, and contradictory ones are discarded. A sufficiency-based stopping mechanism further avoids unnecessary refinement. Experiments on NarraCrime and multiple public
reasoning benchmarks show that EVAR improves both task performance and evidence faithfulness while maintaining controllable inference cost.
Read the full abstract ▾
Computation and Language
You Know What I Mean: A Benchmark for Agentic Conversational Reference Grounding
Karen Fuchs, Uri Katz and Yoav Goldberg
Collaborative conversations frequently contain references whose targets are indirect rather than named: resolving "this looks like the fix discussed yesterday" requires combining conversational context with evidence from the surrounding workspace which is accessible through APIs or user interfaces. We formalize this problem as Conversational Reference Grounding (CoRG): using a given set of tools to resolve a reference in conversation to the unique external item intended by the speaker. CoRG is challenging because it combines lexical, semantic, and temporal cues distributed across the conversation and the external workspace.
Agents must translate these heterogeneous signals into effective tool use: formulating strategies, discovering plausible candidates, inspecting their metadata and content, and ruling out close alternatives. We study CoRG through RepoRef, a
benchmark of 400 developer-chat segments grounded in GitHub issues, pull requests, and commits across 92 repositories. Unlike single-shot
retrieval tasks, RepoRef often requires multi-step tool use. Our results show that CoRG remains challenging for current
agents, even the best
agent reaches only 67.0% success rate, leaving one third of references unresolved. These findings position CoRG as a concrete
benchmark for studying how
agents search, inspect, and verify information in realistic multi-tool environments.
Read the full abstract ▾
Computation and Language
A^2Agent: Action-Aware Reinforcement Learning for Repository-Level Code Localization Agents
Doyeon Kim, Suyoung Bae, Yumin Lee et al. (+1 more)
Localizing issue-relevant code regions is a critical step in automated software engineering. However, due to their reliance on sparse trajectory-level signals, existing methods cannot identify which per-turn actions are effective and often discover correct code regions during exploration but fail to commit them. To address these limitations, we propose an action-aware reinforcement learning method that combines a per-turn reward sequence rewarding both the discovery and commitment of gold code regions with an action-level advantage estimation scheme that isolates each action's credit by grouping turns sharing the same exploration context. Extensive evaluations show that our method improves the average F1 over the state-of-the-art (SOTA) by 1.58% on SWE-Bench Verified and 8.55% on SWE-Bench Pro, with our 4B model outperforming baselines up to 8x larger. Our code is available at https://github.com/donian00/A2
Agent.
Read the full abstract ▾
All authors Doyeon Kim, Suyoung Bae, Yumin Lee, Jee-Hyong Lee
Computation and Language · Cross-listed from cs.CY
Do LLMs Change Their Minds Like Humans? Diagnosing Human–LLM Divergence in Single-Turn Persuasion Judgments
Lin Chen, Yitong Chen and Yong Li
Large
language models (LLMs) are increasingly deployed as proxies for human participants in social simulations, yet whether they update their beliefs in response to persuasive arguments, as humans do, remains poorly understood. We conduct a systematic comparison using a naturally occurring online persuasion corpus in which original posters explicitly verify whether a reply changed their view. Our results show that LLMs achieve only slight agreement with humans (Cohen's kappa ranging from 0.079 to 0.178). Content-level analyses show that humans and LLMs agree on the strongest persuasion cues but diverge on finer ones: humans are more swayed by novel content and assertive
language, whereas LLMs favor topical similarity and surface-level formatting. At the level of persuasion strategy, LLMs underweight emotional appeals and overweight credibility signals relative to humans, while the type of proposition under debate exerts no measurable effect on the degree of divergence. Furthermore, switching from first-person role-playing to third-person observation shifts all models toward greater resistance to persuasion, with the effect varying across persuasion strategies and textual features. These findings highlight the risk of treating LLM judgments as faithful proxies for human belief updating and point to structural differences in how LLMs and humans process persuasive discourse. Our code is available at https://github.com/tsinghua-fib-lab/LLM-belief-update-cmv.
Read the full abstract ▾
Computation and Language
R$^2$A: Learning Persona Policies Through Persona Representation Learning and Runtime Alignment
Mohan Zhang, Chengsong You, Xiaoyu Cao et al. (+4 more)
The same Persona behavior can be beneficial in one context but harmful in another, causing static Persona elicitation to perform inconsistently across tasks. We introduce the Persona Selection–Realization Framework, which models behavior generation through a latent Persona state and decomposes it into Persona Selection and Persona Realization. The discrepancies between static Persona elicitation and an ideal Persona policy in these two components define the Selection Gap and Realization Gap, respectively. Building on this framework, we propose R$^2$A, a two-stage approach for learning Persona policies. Persona Representation Learning uses structured Who–How–What presentations to encode the target Persona's objective, conditional behavioral principles, and trajectory-level manifestations. Persona Runtime
Alignment then removes the explicit Persona specification and jointly calibrates behavior selection and trajectory realization using task feedback. Across 12 evaluation settings covering the four principles of the Accountable-Professional Persona studied in this work, R$^2$A overall outperforms both the base model and static Persona elicitation. Ablation results further show that Persona Representation Learning is critical for preventing Runtime
Alignment from producing behaviorally imbalanced policies and for achieving more stable Persona policy learning.
Read the full abstract ▾
All authors Mohan Zhang, Chengsong You, Xiaoyu Cao, Zhen Sun, Xiaohan Jia, Junwei Zhou, Yongchao Chen
Computation and Language
HiVe: Beyond Static Prompts for Multitask Learning via Hierarchy-based Vertical Mixture-of-Experts
Hyeonjik Bae, Minyeol Kim and Susik Yoon
As large
language models (LLMs) continue to scale, parameter-efficient fine-tuning (PEFT) has become a practical alternative to full-parameter adaptation.
Prompt tuning is effective, but existing approaches either use flat
prompt structures or hierarchical structures with fixed
prompt composition, limiting adaptive
prompt specialization. To address this limitation, we propose HiVe, a
prompt tuning framework that models
prompts at multiple levels and enables input-dependent specialization. HiVe constructs a
prompt hierarchy by leveraging inter-task relationships during training, and employs a vertical mixture-of-experts (V-MoE) mechanism at inference time to compose
prompts up to the level of specialization required for each input. Experiments show that HiVe consistently outperforms strong
prompt tuning baselines across diverse tasks.
Read the full abstract ▾
Computation and Language
ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding
Luxi Lin, Zhanpeng Zeng, Shuang Peng et al. (+2 more)
Speculative decoding accelerates autoregressive
language model inference by having a lightweight draft model propose multiple candidate
tokens, which are then verified in parallel by a larger target model. However, after the first rejection, standard prefix-based verification discards the remaining draft suffix, so the computation spent generating and verifying those positions does not contribute to decoding progress. Focusing on DFlash, we show that rejected positions in a rejected suffix may still align with the target continuation, indicating that the draft model can retain useful semantic and structural information despite local
token-level errors. Motivated by this observation and inspired by conditional
diffusion, we introduce ReTrace, a rejected-trajectory conditioning method that conditions each draft block on the rejected suffix from the previous round rather than generating it from fresh mask placeholders alone. ReTrace retains the hidden representations of the rejected suffixes, aligns them with the next draft block, refines them using target-aware correction signals from the same verification pass, and admits them into the drafter's input
embeddings through gated residual fusion. Because rejected
tokens are never committed and target-side verification remains unchanged, ReTrace preserves the lossless property of speculative decoding without requiring an additional model forward pass. Experiments with Qwen3 models across mathematical
reasoning, code generation, and open-ended dialogue demonstrate that ReTrace consistently improves average acceptance length and end-to-end decoding speed over its DFlash backbone. By introducing cross-round conditioning without modifying within-round proposal generation, ReTrace is largely orthogonal to existing drafting improvements and might be combined with them for further gains.
Read the full abstract ▾
All authors Luxi Lin, Zhanpeng Zeng, Shuang Peng, Songwei Liu, Rongrong Ji
Computation and Language
Evaluating the Capabilities of LLMs for Persuasive Dialogue
Jordan Robinson, Angus R. Williams, Katie Atkinson et al. (+1 more)
Large
language models (LLMs) can generate apparently highly persuasive text, but does sounding persuasive mean arguing well? We introduce Persuasio, a multi-
agent dialogue platform grounded in a formal argumentation-based theory of persuasion dialogues that adjudicates logical winners during free-text debates. Using this system, we generated 192 debates on a UK political topic between humans and LLMs, and evaluated 22 interlocutors through both automated adjudication and 9,702 crowdsourced pairwise judgements across 1{,}386 annotation instances. We observed a consistent decoupling between subjective and formal persuasiveness: LLMs dominated the subjective ranking yet performed substantially worse under argumentation-theoretic adjudication, where humans remained competitive. Multi-
agent and
retrieval-augmented variants further widened this divergence. These findings reveal a systematic gap between rhetorical fluency and formal argumentative strength in LLM-based persuasive dialogues.
Read the full abstract ▾
All authors Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn
Computation and Language · Cross-listed from cs.LG
Higher-Dimensional Rotary Position Embedding
Yixing Li, Ruobing Xie, Yudong Zhang et al. (+3 more)
Transformers rely on position
embedding mechanisms in long context modeling in most cases. Rotary Position
Embedding (RoPE) embeds positional information with independent 2D rotations, forming relative position terms in self-
attention. However, its pairwise, block-based, and decoupled structure limits deep mixing and robustness across channels. We propose HD-RoPE, which extends RoPE from independent 2D rotations to higher-dimensional rotations and introduces a Paley-I orthogonal basis to obtain balanced, isotropic, and dense phase mixing within each rotation subspace. This significantly enhances channel coupling and rotational degrees of freedom while maintaining orthogonal stability and the relative position closure property. Furthermore, HD-RoPE is easily optimized for engineering efficiency without introducing additional trainable parameters. We have conducted extensive evaluation results demonstrating that HD-RoPE achieves significant performance improvements over standard RoPE across various popular
benchmarks and in both long and short contexts.
Read the full abstract ▾
All authors Yixing Li, Ruobing Xie, Yudong Zhang, Yushi Bai, Samm Sun, Yu Cheng
Computation and Language
DVBench: Benchmarking MLLMs for Understanding Dynamic Charts and Narratives in Data Videos
Bomiao Wang, Zekai Shao, Jiexiang Lan et al. (+3 more)
While MLLMs have made significant strides in chart comprehension and
video understanding, current evaluations largely isolate these capabilities, leaving a critical gap in understanding temporally evolving structured visual information. To address this gap, we introduce DVBench, a
benchmark for evaluating MLLMs on data
videos, a storytelling medium that integrates dynamic charts with structured narratives. We decompose data
video understanding into five dimensions. DVBench comprises 300 real-world data
videos and 1,000 human-verified QA pairs curated through a rigorous semi-automated pipeline. Extensive evaluations of nine MLLMs show that Gemini-3.1-Pro achieves the best overall performance, while Kimi-k2.5 is the strongest open-source model. We further identify two notable phenomena: open-source model performance does not scale strictly with parameter size, and narrative proficiency does not guarantee visual capability. Fine-grained analyses and ablation studies further reveal dimension-specific weaknesses and the effects of frame configurations and subtitle inputs, informing future MLLM development. DVBench is publicly available at https://bomiaowang.github.io/DVBench/.
Read the full abstract ▾
All authors Bomiao Wang, Zekai Shao, Jiexiang Lan, Xiaoliang Fu, Xingchen Zeng, Siming Chen
Computation and Language
The Depth Flow of Token Representations Is Nonlinear and Does Not Descend Its Own Density
Alexandre Quemy
A
token's representation is carried through the network layer by layer. The whole vocabulary carried together forms a flow. We fit this flow's equation of motion as a discrete Langevin model over corpus-mean trajectories of Pythia-160M and Pythia-410M, and score the predicted steps on held-out
tokens. Linear maps are often used as cheap surrogates for a layer. The flow they summarize is not linear: a quadratic drift beats the linear linear map at every transition of both models, and the Kramers–Moyal
estimator agrees wherever its neighborhoods stay local. We then characterize the flow further. First, we show that it does not descend its own log-density. The drift instead descends a potential that is not the density. Second, the rotational component is not negligible, $4$ to $45\%$ of the explainable drift, and the circulation shows in what the flow preserves: a
token keeps its angular rank across all thirteen layers while its norm rank is shuffled and its concentration rank is reversed by the last block.
Read the full abstract ▾
Computation and Language
A Hub of Short Rows Inflates Intrinsic Dimension Estimation of Token Embeddings
Alexandre Quemy
A
token-
embedding table holds a hub of short rows near its origin, and we show that this cluster biases what nearest-neighbor intrinsic-dimension (ID)
estimators report. Because of the concentration of measure, a
token is closer to the central cluster than to any other
token, so its first two neighbors are both hub rows at nearly the same distance. As a result, the ID
estimators such as TwoNN return a dimension far above the real ID. Measured one
token at a time, dimension is a heavy-tailed distribution. Measured on the full vocabulary, it grows with the model's parameter count. However, when we remove the hub, the heavy tail disappears and the measured dimension collapses to a narrow range for eleven models, from GPT-2 to models such as K3 and GLM-4.7. The hub acts as a switch: a few hundred rows are enough to fully inflate the estimate. We reproduced an experiment stating that the intrinsic dimension (ID) of Pythia's
token-
embedding table grows with the parameter count, from $27$ to $122$ between 160M and 12B parameters. We show that this result disappears when the hub is removed: the table then reads $10$ to $17$ at every size. The hub contains a subset of the population that under-trained-
token detectors flag, but on Pythia the hub that we detected and removed as a whole was updated during training: what seem to characterize these rows is simply their length, not an absence of updates. Finally, we show that normalizing the rows instead of removing them gives the same lower reading.
Read the full abstract ▾
Computation and Language
ACTD: Anchor-Based Cross-Tokenizer Distillation with Residual Regularization
Huiyi Zhang, Zijian Li, Xiaocheng Feng et al. (+4 more)
Knowledge
distillation effectively transfers
reasoning capabilities from large
language models to lightweight student models. To enable knowledge transfer across disparate model families, researchers increasingly explore cross-tokenizer
distillation. However, cross-tokenizer
distillation remains challenging due to vocabulary and sequence misalignment, while approximate vocabulary
alignment can introduce additional noise into
distillation. To address these challenges, we propose Anchor-Based Cross-Tokenizer
Distillation with Residual Regularization (ACTD). ACTD bridges structural heterogeneity through vocabulary and sequence
alignment, while mitigating
alignment noise via a novel anchor loss with residual regularization. We further extend this framework to a multi-teacher setting. Evaluated across five
reasoning benchmarks with three distinct teacher models, ACTD achieves state-of-the-art performance. Moreover, its multi-teacher extension outperforms the strongest single-teacher and multi-teacher baselines, further demonstrating the robustness of our method.
Read the full abstract ▾
All authors Huiyi Zhang, Zijian Li, Xiaocheng Feng, Weitao Ma, Xiaoliang Yang, Yichong Huang, Bing Qin
Computation and Language · Cross-listed from cs.CV
Conducting Stylistic Analysis of Paintings through an Art-History Agent
Marc S. Walton and Astrid Harth
Attributing an artwork to an artist has traditionally relied on detailed visual observations and descriptions, known as stylistic analysis in art history. By contrast, current artificial intelligence (AI) models used in the field offer only unexplained probabilistic classifications. To bridge this methodological gap, we present an AI framework that automates stylistic analysis of paintings, providing a foundation for enhancing evidence collection, discovery, and verification. By training a
vision transformer (ViT) on a large corpus of paintings with metadata, our system encodes this art history-specific data as
embeddings. These representations are factorized via sparse dictionary learning into a shared set of features that recur across the training set. A large
language model (LLM) then interprets each feature by retrieving associated artworks and their accompanying curator-written texts, and synthesizes them into descriptions that reflect their stylistic attributes. Finally, an autonomous coordinator LLM applies a
reasoning-and-action (ReAct) framework to weight, test, and refine these features into cohesive descriptions of an artwork, or comparisons of artworks. This approach converts detailed visual features into descriptive terms, addressing a key challenge in art history. It thus connects the use of images as data with the semantic concerns of humanists, establishing
vision-based computational art history as an area for future growth.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SE
InteractBench: Benchmarking LLMs on Competitive Programming under Unrevealed Information
Jiaze Li, Aocheng Shen, Bing Liu et al. (+4 more)
Competitive programming is increasingly being used to evaluate the algorithmic
reasoning capabilities of large
language models (LLMs). However, existing
benchmarks primarily focus on full-information tasks where all problem inputs are provided upfront. This overlooks a critical dimension of algorithmic
reasoning: the ability of generated programs to operate when key information is not revealed upfront. Interactive problems, a distinctive component of competitive programming, embody this challenge. These problems require programs to engage in multi-round interaction with an interactor (a judge program) under strict protocol constraints and limited query budgets, with new information revealed only in response to queries. To address this gap, we introduce InteractBench, a
benchmark comprising 322 high-quality interactive problems curated from Codeforces, AtCoder, IOI, and ICPC. Each problem is packaged with executable local interactors, enabling fully offline evaluation. Unlike existing
benchmarks, InteractBench assesses whether model-generated code can acquire information and track state dynamically. Our evaluation reveals a significant interaction gap: even the most advanced
reasoning models achieve limited success on interactive problems. Beyond success rates, we propose a fine-grained failure taxonomy to diagnose the root causes of these deficiencies. Although algorithmic logic errors remain dominant, protocol violations and query-budget overruns are frequent. Code is available at https://github.com/kmsgk0/InteractBench.
Read the full abstract ▾
All authors Jiaze Li, Aocheng Shen, Bing Liu, Boyu Zhang, Xiaoxuan Fan, Qiankun Zhang, Xianjun Deng
Computation and Language
PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatization
Stephen Meisenbacher, Andreea-Elena Bodea, Ahmet Bilal Ak{\i}n et al. (+3 more)
Natural
Language Processing methods have enabled novel solutions and advances in the field of
privacy, particularly in the sub-domain of text-to-text privatization, where the goal is to transform a sensitive input text into a privatized output by ideally masking (in)directly identifiable or otherwise private information. The evaluation of text-to-text privatization, however, is not straightforward, and the extant literature has utilized a myriad of techniques and metrics to quantify the
privacy-preserving capabilities of privatization methods. Seeking to unify the evaluation of text-to-text privatization, we introduce PrivBench, a holistic and modular benchmarking platform for researchers and practitioners working on text privatization. PrivBench is holistic in that it evaluates privatization on a series of defined desiderata, which are structured into modules. PrivBench is not only modular but also extensible, allowing for future updates and
benchmark versions. PrivBench is user-centered and promotes competition via real-time evaluation and a live public leaderboard. The platform is free to use and openly accessible at https://privbench.com/.
Read the full abstract ▾
All authors Stephen Meisenbacher, Andreea-Elena Bodea, Ahmet Bilal Ak{\i}n, Alexandra Klymenko, Jana Diesner, Florian Matthes
Computation and Language
MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation
Yangsong Lan, Renkai Hu, HongKai Zheng et al. (+4 more)
Recent advances in large
reasoning models (LRMs) have shown strong performance on complex problems through long chain-of-thought (Long CoT)
reasoning. However, distilling such trajectories into smaller student models remains challenging: direct Long CoT supervision often provides limited gains and can be less effective than concise Short CoT rationales. In this work, we investigate this phenomenon from a gradient-centric perspective. Our analysis shows that Long CoT induces larger gradient magnitudes and more concentrated update directions than Short CoT, with this effect becoming more pronounced as student model capacity increases. These findings suggest that effective Long CoT
distillation requires balancing the
reasoning information density of
reasoning trajectories with their distributional
alignment to the student model. Motivated by this insight, we propose Model Interporlation
Distillation (MI-
Distillation), a framework that constructs a continuous Instruct-
Reasoning data spectrum through model interpolation. To select suitable trajectories from this spectrum, we further introduce Sequential Learnable Surprisal Score (SeqLSS), which favors
reasoning paths that are both informative and learnable for the student. Extensive experiments on
reasoning benchmarks show that MI-
Distillation consistently improves small model CoT
distillation over strong Long CoT baselines.
Read the full abstract ▾
All authors Yangsong Lan, Renkai Hu, HongKai Zheng, Bo Zhang, Renzhi Wang, Hongliang Dai, Piji Li
Computation and Language
Memory-First Fact-Checking: A Knowledge-Graph-Grounded Multi-Agent System for Misinformation Detection
Amelia Petrenciuc, Alexandru Lecu and Adrian Groza
This paper introduces a hybrid fact-checking framework that integrates Knowledge Graph-based semantic
memory with
adversarial multi-
agent reasoning for explainable misinformation detection. The proposed system follows a
memory-first, web-fallback architecture, in which input claims are initially evaluated against a dual-index Knowledge Graph through Sentence-BERT-based semantic
retrieval and Natural
Language Inference. When the evidence retrieved from the graph is insufficient to support a reliable decision, the framework collects information from trusted web sources and assesses it using an
adversarial tribunal composed of support, contradiction, and judging
agents. A graph-aware confidence mechanism combines semantic similarity, NLI confidence, and structural graph evidence to determine whether internal knowledge is sufficient, thereby reducing unnecessary web
retrieval. Following verification, validated information is transformed into structured triples and incorporated into the Knowledge Graph, supporting the incremental expansion of the system's semantic
memory. Experimental evaluation on a curated COVID-19 misinformation
benchmark demonstrates that the proposed framework achieves an accuracy of 97.4% and a macro-averaged F1-score of 92.6% on resolved claims, outperforming a Llama~3.3~70B baseline, which obtains an accuracy of 87.7% and a macro-averaged F1-score of 86.3%.
Read the full abstract ▾
Computation and Language
JPO: Juris Policy Optimization for Structured Legal Reasoning in Criminal Judgment Prediction
Zhaolu Kang, Yantao Liu, Tailong Luo et al. (+14 more)
Criminal judgment prediction requires models to infer statutory articles, charges, and sentencing outcomes from case facts. Unlike standard classification tasks, it involves a structured
reasoning process in which statutes should be matched with facts, charges should be justified by statutes, and sentencing outcomes should remain consistent with charges. Existing approaches optimize final labels, and while some have attempted to evaluate
reasoning quality, their evaluations are indirect, often relying on LLM-generated rubrics that reflect model-internal preferences rather than the inherent logical structure of legal adjudication. We propose Juris Policy Optimization (JPO), a post-training framework for structured legal
reasoning in Chinese criminal judgment prediction. JPO first uses teacher-generated rationales to supervise a standardized four-step
reasoning process, and then applies reinforcement learning with a composite reward over legal prediction quality,
reasoning structure completeness, and cross-step consistency. JPO further introduces
token-level advantage reweighting and adaptive clipping for legally salient
reasoning segments. Experiments on multiple open-source
language models and three Chinese legal
benchmarks show that JPO consistently improves both judgment prediction and
reasoning quality over supervised fine-tuning and reinforcement learning baselines.
Read the full abstract ▾
All authors Zhaolu Kang, Yantao Liu, Tailong Luo, Leqi Zheng, Lei Wei, Chenghua Zhu, Junhao Gong, Jiachen Qian, Eric Hanchen Jiang, Jiaxin Liu, Yuan Wang, Hao Zhang, Zixia Wang, Rong Fu, Zheng Lin, Richeng Xuan, Zhichao Hu
Computation and Language
Cross-lingual Functional Vectors for Emotion Detection in Large Language Models
Jieying Xue, Phuong Minh Nguyen, Minh Le Nguyen et al. (+1 more)
Function vectors (FVs) have recently emerged as a promising mechanism for steering the behavior of large
language models (LLMs) by injecting task-specific latent direction representations derived from in-context demonstrations. While prior studies have shown that FVs can recover task behavior in structured in-context learning settings, their effectiveness on semantically complex tasks and their ability to generalize across
languages remain underexplored. We investigate the cross-lingual transferability of FVs using
multilingual multi-label
emotion recognition as a challenging semantic classification
benchmark. Specifically, we examine whether FVs extracted from a source
language can steer task behavior in another
language under both standard clean and perturbed zero-shot settings without providing demonstrations during inference. Across diverse cross-lingual settings, applying FVs substantially improves performance, suggesting that FVs capture
language-agnostic, task-relevant signals rather than purely
language-specific lexical patterns, and highlighting their potential as a lightweight and transferable mechanism for
multilingual task adaptation. We observe that each LLM exhibits a relatively stable optimal range of
attention heads for constructing effective FVs, and the pattern remains consistent across
languages. In addition, FVs can partially replicate the task-steering effects of standard few-shot in-context learning while avoiding the computational overhead of processing multiple demonstrations, making them effective for large-scale practical applications. Our code is available at https://github.com/yingjie7/cross_lingual_fvs.
Read the full abstract ▾
All authors Jieying Xue, Phuong Minh Nguyen, Minh Le Nguyen, Shogo Okada
Computation and Language
Beyond Surface Alignment: Grounding the Dynamics of Situational Understanding and Generative Control in LLMs
Chenghao Yang
The current
alignment tuning paradigm for Large
Language Models (LLMs) prioritizes surface-level behaviors – fluency, safety, and tonal consistency. While effective for casual chat, this thesis argues that such surface
alignment masks a lack of grounding, creating models that are stylistically confident but situationally brittle. We propose a framework of Grounded
Alignment, analyzing how models process context (Input) and structure generation (Output), then aligning these grounded behaviors to human needs. First, we evaluate failures in Situational Grounding. SitTest shows that despite large context windows, state-of-the-art models struggle to maintain a consistent "mental model" of a changing environment. ReCode further shows that models rely on surface heuristics rather than deep syntactic dependencies: they "read" extensive histories without truly "understanding" the evolving situation. Second, we evaluate Generative Grounding. We introduce the Branching Factor (BF) to map LLM generation, finding that standard
alignment tuning constricts this landscape into premature stylistic collapse. Hindsight further shows that models often fail to understand their own generations. Finally, we propose Dynamic Control for grounded interaction. AI Realtor demonstrates context engineering to compensate for poor situational grounding. Base-Aligned Model Collaboration decouples exploration from stylistic constraints. We also present Annealed Sampling for verifiable reinforcement learning and apply these ideas to Addiction Support, where model-generated rationalization offers a communication interface for high-stakes domains. Collectively, this work moves beyond surface
alignment toward
agents anchored in both context and generation.
Read the full abstract ▾