Computation and Language · Cross-listed from cs.LG
Dimitrios Damianos, Leon Voukoutis, Georgios Skyrianos et al. (+2 more)
Generative
Vision-
Language Models (VLMs) perform well on
multimodal reasoning, but how visual inputs are transformed to text remains poorly understood. Existing interpretability work on VLMs uses Sparse Autoencoders (SAEs), which decompose static residual representations and miss the functional updates that drive cross-modal interaction. We adopt a function-centric framework based on Transcoders, sparse approximations of MLP sublayers that act as a causal proxy for layer-wise computation. Applied to Gemma 3-4B-IT, the framework decomposes the model into interpretable computational pathways linking image patches to directions in
token generation. Transcoder attributions produce stronger and more stable effects on visually grounded
tokens under patch ablation than SAE attributions, and align better with semantically relevant image regions. A False Visual Grounding counterfactual analysis confirms that the recovered pathways are specific to
vision-
language interaction.Finally, we perform a structural analysis of hallucinated generations, by extracting graph-based indicators from
circuit traces produced by the transcoders. A logistic classifier over these mechanistic graph features predicts
hallucinations at AUC $0.68$. These results show that function-centric
circuit decomposition yields interpretable and predictive accounts of
multimodal computation in VLMs.
Read the full abstract ▾
All authors Dimitrios Damianos, Leon Voukoutis, Georgios Skyrianos, Vassilis Katsouros, Georgios Paraskevopoulos
Computation and Language · Cross-listed from cs.AI
ImProver 2: Iteratively Self-Improving LMs for Neurosymbolic Proof Optimization
Riyaz Ahuja, Tate Rowney, Jeremy Avigad et al. (+1 more)
Formal mathematics libraries are rapidly expanding, creating a growing need to refactor verified proofs for maintainability and to improve training data quality for neural provers. However, scalable proof optimization is hindered by heterogeneous and heuristically specified objectives, scarce data, and high training and inference costs. To overcome these challenges, we introduce ImProver 2, a neurosymbolic framework for automated proof optimization in Lean 4. ImProver 2 combines a data-efficient expert-iteration pipeline with a scaffold that exposes formal structure alongside lightweight informal abstractions. We further introduce a suite of metrics capturing structural proof properties. Using ImProver 2, we train a 7B-parameter model that outperforms orders-of-magnitude larger models within the same model family, and is competitive with mid-tier frontier models across metrics. We additionally demonstrate that our neurosymbolic scaffold significantly improves performance across both small and frontier models. We show that with proper scaffolding and training, small models can effectively restructure research-level proofs over complex and varied metrics, matching substantially larger systems and establishing proof optimization as a scalable, learnable task.
Read the full abstract ▾
All authors Riyaz Ahuja, Tate Rowney, Jeremy Avigad, Sean Welleck
Computation and Language
Tokenisation via Convex Relaxations
Jan Tempus, Philip Whittington, Craig W. Schmidt et al. (+2 more)
Tokenisation is an integral part of the current NLP pipeline. Current tokenisation
algorithms such as BPE and Unigram are greedy
algorithms – they make locally optimal decisions without considering the resulting vocabulary as a whole. We instead formulate tokeniser construction as a linear program and solve it using convex optimisation tools, yielding a new
algorithm we call ConvexTok. We find ConvexTok consistently improves intrinsic tokenisation metrics and the bits-per-byte (BpB) achieved by
language models; it also improves downstream task performance, but less consistently. Furthermore, ConvexTok allows the user to certify how far their tokeniser is from optimal, with respect to a certain objective, via a lower bound, and we empirically find it to be within 1% of optimal at common vocabulary sizes.
Read the full abstract ▾
All authors Jan Tempus, Philip Whittington, Craig W. Schmidt, Dennis Komm, Tiago Pimentel
Computation and Language · Cross-listed from cs.LG
Vector Policy Optimization: Training for Diversity Improves Test-Time Search
Ryan Bahlous-Boldi, Isha Puri, Idan Shenfeld et al. (+6 more)
Language models must now generalize out of the box to novel environments and work inside inference-scaling search procedures, such as AlphaEvolve, that select rollouts with a variety of task-specific reward functions. Unfortunately, the standard paradigm of LLM post-training optimizes a pre-specified scalar reward, often leading current LLMs to produce low-
entropy response distributions and thus to struggle at displaying the diversity that inference-time search will require. We propose Vector Policy Optimization (VPO), an RL
algorithm that explicitly trains policies to anticipate diverse downstream reward functions and to produce diverse solutions. VPO exploits that rewards are often vector-valued in practice, like per-test-case correctness in code generation or, say, multiple different user personas or reward models. VPO is essentially a drop-in replacement for the GRPO advantage
estimator, but it trains the LLM to output a set of solutions where individual solutions specialize to different trade-offs in the vector reward space. Across four tasks, VPO matches or beats the strongest scalar RL baselines on test-time search (e.g. pass@k and best@k), with the gap widening as the search budget grows. For evolutionary search, VPO models unlock problems that GRPO models cannot solve at all. As test-time search becomes more standardized, optimizing for diversity may need to become the default post-training objective.
Read the full abstract ▾
All authors Ryan Bahlous-Boldi, Isha Puri, Idan Shenfeld, Akarsh Kumar, Mehul Damani, Sebastian Risi, Omar Khattab, Zhang-Wei Hong, Pulkit Agrawal
Computation and Language
Evaluating Commercial AI Chatbots as News Intermediaries
Mirac Suzgun, Emily Shen, Federico Bianchi et al. (+5 more)
AI chatbots are rapidly shaping how people encounter the news, yet no prior study has systematically measured how accurately these systems, with their proprietary search integrations and
retrieval-synthesis pipelines, handle emerging facts across
languages and regions. We present a 14-day (February 9-22, 2026) evaluation of six AI chatbots (Gemini 3 Flash and Pro, Grok 4, Claude 4.5 Sonnet, GPT-5 and GPT-4o mini) on 2,100 factual questions derived from same-day BBC News reporting across six regional services (US & Canada, Arabic, Afrique, Hindi, Russian, Turkish). The best systems achieve over 90% multiple-choice accuracy on questions about events reported hours earlier. The same systems, however, lose 11-13% under free-response evaluation, and 16-17% across the cohort. We further characterize three failure patterns. First, every model achieves its lowest accuracy on Hindi (79% vs. 89-91% elsewhere) and citations indicate an Anglophone
retrieval bias (e.g., models answering Hindi queries cite English Wikipedia more than any Hindi outlet). Second,
retrieval, not
reasoning, failures drive over 70% of all errors. When models retrieve a correct source, they often extract the correct answer; the problem is to land on the right source in the first place. Third, models achieving 88-96% accuracy on well-formed questions drop to 19-70% when questions contain subtle false premises, with the most vulnerable model accepting fabricated facts 64% of the time. We also identify a detection-accuracy paradox: the best false-premise detector ranks second in
adversarial accuracy (abstention rate), while a weaker detector ranks first, showing that premise detection and answer recovery are partially independent capabilities. Overall, these suggest that high accuracy can mask systematic regional inequity, near-total dependence on
retrieval infrastructure, and vulnerability to imperfect queries real users pose.
Read the full abstract ▾
All authors Mirac Suzgun, Emily Shen, Federico Bianchi, Alexander Spangher, Thomas Icard, Daniel E. Ho, Dan Jurafsky, James Zou
Computation and Language
Reducing Political Manipulation with Consistency Training
Long Phan, Devin Kim, Alexander Pan et al. (+3 more)
Large
language models (LLMs) exhibit systematic political bias across a variety of sensitive contexts. We find that LLMs handle counterpart topics from opposing political sides asymmetrically. We refer to this phenomenon as covert political bias and identify 7 categories of techniques through which it operates. We propose two metrics for covert bias: Sentiment Consistency measures
symmetry in rhetoric and framing across paired political
prompts; Helpfulness Consistency measures symmetric depth and engagement. To reduce both types of covert bias, we introduce Political Consistency Training (PCT), an RL training method with two complementary paradigms: Sentiment Consistency Training and Helpfulness Consistency Training. We show that PCT preserves overall helpfulness, substantially reduces covert political bias, and generalizes to held-out
benchmarks. We release our work at https://political-manipulation.ai
Read the full abstract ▾
All authors Long Phan, Devin Kim, Alexander Pan, Alice Blair, Adam Khoja, Dan Hendrycks
Computation and Language
Understanding Data Temporality Impact on Large Language Models Pre-training
Hippolyte Pilchen, Romain Fabre, Franck Signe Talla et al. (+2 more)
Large
language models (LLMs) are typically trained on shuffled corpora, yielding models whose knowledge is frozen at train time and whose temporal grounding remains poorly understood. In this work, we study the impact of pre-training dynamics on the acquisition of time-sensitive factual knowledge, focusing specifically on data ordering. Our main contributions are twofold. First, we introduce a comprehensive
benchmark of over 7,000 temporally grounded questions and an evaluation protocol that enables analysis of whether models correctly associate facts with their corresponding time periods. Second, we pretrain 6B-parameter models on temporally ordered Common Crawl snapshots and compare them against standard shuffled pre-training. Our results show that sequentially trained models match shuffled baselines on general
language understanding and common knowledge while consistently exhibiting more up-to-date and temporally precise knowledge. Temporally ordered pre-training yields improved factual freshness, while shuffled pre-training peaks on older data, possibly due to increased factual repetition. These findings, along with the release of our code at https://github.com/kyutai-labs/kairos , checkpoints, and
datasets at https://huggingface.co/collections/kyutai/kairos provide a foundation for future research on continual learning for LLMs.
Read the full abstract ▾
All authors Hippolyte Pilchen, Romain Fabre, Franck Signe Talla, Patrick Perez, Edouard Grave
Computation and Language
ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning
Md Shamim Ahmed, Farzaneh Firoozbakht, Lukas Galke Poech et al. (+2 more)
Biomedical knowledge graphs (KGs) treat disease associations as static facts, but temporal information is crucial for clinical
reasoning, e.g., a symptom diagnostic of one disease at age 3 may imply a different disease at age 13. Existing KGs such as PrimeKG, Hetionet, and iKraph do not encode when a finding becomes clinically relevant over the course of a disease. This limits their usefulness for longitudinal clinical
reasoning and
retrieval augmentation. We introduce ChronoMedKG, a temporal biomedical knowledge graph that contains 460,497 evidence-linked triples (filtered from 13M raw extractions) covering 13,431 diseases. Each association is tied to temporal components like onset window or progression stage, which are backed by PMID-traceable evidence and a multi-signal credibility score. The graph is constructed through a disease-autonomous multi-
agent pipeline in which multiple frontier LLMs independently extract knowledge from PubMed and PMC literature. Only those relations are kept that are supported by multi-model consensus, survive credibility filtering, as well as ontology
alignment. ChronoMedKG scored 92.7% agreement against Orphadata and adds temporal grounding for 6,250 diseases absent from HPOA, Orphadata, and Phenopackets, including 1,657 Orphanet-coded rare diseases. We further introduce ChronoTQA, a
benchmark of 3,341 questions across eight task types (six temporal plus two static controls), with a 12-question supplementary probe. Frontier LLMs lose roughly 30 points moving from static to temporal questions; ChronoMedKG
retrieval rescues 47-65% of their long-tail failures, against 17-29% for HPOA-RAG. As such, ChronoMedKG provides a crucial temporal axis for
retrieval-augmented clinical systems that was previously absent.
Read the full abstract ▾
All authors Md Shamim Ahmed, Farzaneh Firoozbakht, Lukas Galke Poech, Jan Baumbach, Richard Röttger
Computation and Language · Cross-listed from cs.AI
Beyond Acoustic Emotion Recognition: Multimodal Pathos Analysis in Political Speech Using LLM-Based and Acoustic Emotion Models
Juergen Dietrich
We investigate whether acoustic
emotion recognition models can serve as proxies for the Pathos dimension in political
speech analysis, as operationalised by the TRUST multi-
agent large
language model (LLM) pipeline. Using a Bundestag plenary
speech by Felix Banaszak (51 segments, 245 s) as a case study, we compare three analysis modalities: (1)
emotion2vec_plus_large, an acoustic
speech emotion recognition (SER) model whose continuous Arousal and Valence values are derived via post-hoc Russell Circumplex projection; (2) Gemini 2.5 Flash, an LLM analysing the full
speech audio together with its transcript in an open-ended, context-aware fashion; and (3) TRUST-Pathos scores from a three-advocate LLM supervisor ensemble. Spearman rank correlations reveal that Gemini Valence correlates strongly with TRUST-Pathos (rho = +0.664, p < 0.001), whereas
emotion2vec Valence does not (rho = +0.097, p = 0.499). We further demonstrate, via a systematic quality evaluation of the Berlin Database of Emotional
Speech (EMO-DB) using Gemini in an open-ended annotation paradigm, that standard SER
benchmark corpora suffer from acted
speech, cultural bias, and category incompatibility. Our results suggest that LLM-based
multimodal analysis captures semantically defined political
emotion substantially better than acoustic models alone, while acoustic features remain informative for low-level Arousal estimation. Future work will extend this approach to
video-based analysis incorporating facial expression and gaze.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild
Baiyu Chen, Zechen Li, Wilson Wongso et al. (+5 more)
As wearable and mobile devices become increasingly embedded in daily life, they offer a practical way to continuously sense human motion in the wild. But inertial signals are highly dependent on the sensing setup, including body location, mounting position, sensor orientation, device hardware, and sampling protocol. This setup dependence makes it difficult to learn motion representations that transfer across devices and
datasets, and limits the broader use of wearable IMUs beyond closed-set recognition. We introduce AnyMo, a geometry-aware framework for setup-agnostic human motion modeling. AnyMo uses physics-grounded IMU simulation over dense body-surface placements to generate diverse and plausible synthetic signals, pre-trains a graph encoder from paired synthetic placement views and masked partial observations, tokenizes multi-position IMU into full-body motion
tokens, and aligns these
tokens with an LLM for motion-
language understanding. We evaluate AnyMo on three complementary tasks: zero-shot activity recognition across 14 unseen downstream
datasets, cross-modal
retrieval, and wearable IMU motion captioning, where it improves average Accuracy/F1/R@2 by 11.7%/11.6%/22.6% on HAR, increases zero-shot IMU-to-text and text-to-IMU
retrieval MRR by 15.9% and 28.6%, respectively, and improves zero-shot captioning BERT-F1 by 18.8%. These results support AnyMo as a generalist model for wearable motion understanding in the wild. Project page: https://baiyuchen.com/project/AnyMo.
Read the full abstract ▾
All authors Baiyu Chen, Zechen Li, Wilson Wongso, Lihuan Li, Xiachong Lin, Hao Xue, Benjamin Tag, Flora Salim
Computation and Language · Cross-listed from cs.AI
AMEL: Accumulated Message Effects on LLM Judgments
Sid-Ali Temkit
Large
language models are routinely used as automated evaluators: to review code, moderate content, or score outputs, often with many items passing through one conversation. We ask whether the polarity of prior conversation history biases subsequent judgments, an effect we call the accumulated message effect on LLM judgments (AMEL). Across 84,088 API calls to 12 models from 5 providers (OpenAI, Anthropic, Google, DeepSeek, and four open-source models), we present identical test items in isolation or following histories saturated with predominantly positive or negative evaluations. Models shift toward the conversation's prevailing polarity (d = -0.17, p < 10^-53). The effect concentrates on items where the model is genuinely uncertain at baseline (d = -0.36 for high-
entropy items, vs d = -0.15 when the baseline is deterministic). Bias does not grow with context length: 5 prior turns and 50 produce the same shift (Spearman |r| < 0.01; OLS slope p = 0.80). And there is a negativity asymmetry: paired per item, negative histories induce 1.52x more bias than positive (t = 13.03, p < 10^-36, n = 2,733). Scaling helps but does not solve it (Anthropic: Haiku -0.22 to Opus -0.17; OpenAI: Nano -0.34 to GPT-5.2 -0.17). Three follow-ups narrow the mechanism. The
token probability distribution shifts continuously, not at a threshold. The negativity asymmetry has both
token-level and semantic components, though attributing the balance is exploratory at our sample sizes. Position does not matter: five biased turns anywhere in a 50-turn history produce the same shift. The simplest fix for evaluation pipelines is a fresh context per item; when batching is unavoidable, balancing the history helps.
Read the full abstract ▾
Computation and Language
Tokenization with Split Trees
Craig W. Schmidt, Michael Krumdick, Adam Wiemerslage et al. (+4 more)
We introduce Tokenization with Split Trees (ToaST), a subword tokenization method that directly optimizes compression under a new recursive inference procedure. ToaST greedily splits each pretoken into a full binary tree using precomputed byte n-gram counts, independent of any vocabulary. Given a vocabulary, inference recursively descends each split tree and emits the first in-vocabulary node reached on each path. Vocabulary selection is formulated as an Integer Program (IP) that minimizes the total
token count over all split trees under this inference procedure. The Linear Programming (LP) relaxation is near-integral in practice, yielding provably near-optimal vocabularies, with training time empirically scaling quadratically in the number of split trees. On English text, ToaST reduces
token counts by more than 11% compared to BPE, WordPiece, and UnigramLM at vocabulary sizes of 40,960 and above, reducing the number of inference
tokens for models using this tokenizer, thus extending the effective context length. ToaST also uses common single-byte
tokens less frequently than these baselines, leading to a substantial improvement in Renyi efficiency. In experiments training 1.5B parameter
language models, ToaST achieves the highest CORE score, outperforming baselines by 2.6%–7.6%, with significance for two of three, and scoring best on 13 of 22 individual tasks.
Read the full abstract ▾
All authors Craig W. Schmidt, Michael Krumdick, Adam Wiemerslage, Seth Ebner, Varshini Reddy, Yuval Pinter, Chris Tanner
Computation and Language
Self-Policy Distillation via Capability-Selective Subspace Projection
Guangya Hao, Hanxue Liang, Yitong Shang et al. (+2 more)
Self-
distillation bootstraps large
language models (LLMs) by training on their own generations. However, existing methods either rely on external signals to curate self-generated outputs (e.g., correctness filtering, execution feedback, and reward search), which are costly and unavailable for the best-performing frontier models, or skip curation entirely and train on all raw outputs, an approach that is often domain-specific and hard to generalize. Both also share a deeper weakness that self-generated outputs entangle task-relevant capability with others, such as stylistic patterns, formatting artifacts, and model-specific errors, diluting the signal for the specific capability one aims to improve. In this paper, we propose Self-Policy
Distillation (SPD), which achieves generalizable, capability selective with only minimal external signal (<5% labeled data in each domain). Specifically, SPD extracts a low-rank capability subspace from the model's own gradients on correctness-defining
tokens, projects key-value (KV) activations into this subspace during self-generation, and fine-tunes on the resulting raw outputs with standard next-
token prediction loss.Through extensive experiments across code generation, mathematical
reasoning, and multiple-choice QA, we show that SPD achieves up to 13% improvement over state-of-the-art self-
distillation methods and up to 16.3% improvement over pre-trained baselines. Notably,SPD demonstrates superior generalizability, achieving 15.7% better performance under out-of-domain generalization settings.
Read the full abstract ▾
All authors Guangya Hao, Hanxue Liang, Yitong Shang, Yunbo Long, Zhuokai Zhao
Computation and Language
Moral Semantics Survive Machine Translation: Cross-Lingual Evidence from Moral Foundations Corpora
Maciej Skorski
Moral
language is subtle and culturally variable, making it difficult to translate faithfully across
languages. Idiomatic expressions, slang, and cultural references introduce hard-to-avoid translation artifacts. Yet automated moral values classification depends on
language-specific annotated corpora that exist almost exclusively in English. We investigate whether LLM-based translation can bridge this gap, taking Polish as a test case. Using ~50k morally-annotated social media posts from a diverse range of topics, we apply a principled four-method validation pipeline: LaBSE cross-lingual
embedding similarity, Centered Kernel
Alignment (CKA), LLM-as-judge evaluation, and deep learning classifier parity tests. We show that despite shortcomings in handling slang, vulgarity, and culturally-loaded expressions, direct translation preserves subtle moral cues well enough to be harvested by cross-lingual machine learning - with a mean cosine similarity of 0.89 and classification accuracy gaps of 0.01-0.02 AUROC across foundations. These results demonstrate that machine translation is a practical and cost-effective path to moral research in
languages currently under-resourced in this domain. We demonstrate this for Polish as a representative Slavic
language, with expected generalization to related
languages.
Read the full abstract ▾
Computation and Language
Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs
Shanshan Wang, Fengying Ye, Hanjia Lyu et al. (+6 more)
Previous detection studies have shown that LLMs cannot be effectively used as detectors, but these studies have not addressed modern Chinese poetry. Moreover, no relevant research has explored the performance of LLMs in detecting modern Chinese poetry. This paper evaluates and enhances the performance of LLMs as detectors for modern Chinese poetry, and proposes an image-semantic guided poetry detection method. Compared with traditional detection approaches, our method innovatively incorporates images that reflect the content of the poetry. Through example-driven approaches, our method effectively integrates information such as meaning, imagery, and feeling from the image, then forms a complementary judgment with the poem text. Experimental results demonstrate that the LLM detectors based on our method outperform baseline detectors based on plain text, and even surpass the best-performing traditional detector, RoBERTa. The Gemini detector using our method achieves a Macro-F1 score of 85.65%, reaching the state-of-the-art level. The performance improvements of different LLM detectors on multiple LLMs-generated data prove the effectiveness of our method.
Read the full abstract ▾
All authors Shanshan Wang, Fengying Ye, Hanjia Lyu, Caiwen Gou, Junchao Wu, Jingming Yao, Chengzhong Xu, Jiebo Luo, Derek F. Wong
Computation and Language
Whose Voice Counts? Mapping Stakeholder Perspectives on AI Through Public Submissions to the U.S. Government
Alina Karakanta, Alex Christiansen, Tomás Dodds et al. (+4 more)
As artificial intelligence (AI) systems become more common in our daily lives, it is important to understand how different stakeholders comprehend and envisage the role that these technologies play in shaping social, political, and economic realities. In this paper, we investigate public perceptions of AI based on a corpus of letters submitted during the public consultation for the Trump Administration's US AI Action Plan. To this aim, we release a corpus cleaning pipeline and perform topic modelling and frequency analysis to explore predominant topics discussed by different subgroups (e.g., academia, individuals, private sector) and those appearing in the AI Action Plan. Our results show that individuals voice strong concerns related to the impact of AI on life, while other stakeholders are more concerned with AI development. Our comparison of topics suggests that the AI Action Plan reflects predominantly the concerns of the private sector on security, policies, and development, with individuals' concerns less represented.
Read the full abstract ▾
All authors Alina Karakanta, Alex Christiansen, Tomás Dodds, Bissie Anderson, Matteo Fuoli, Marcus Perlman, Aletta G. Dorst
Computation and Language
Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety
Piercosma Bisconti, Matteo Prandi, Federico Pierucci et al. (+11 more)
Background. Traditional safety
benchmarks for
language models evaluate generated text: whether a model outputs toxic
language, reproduces bias, or follows harmful instructions. When models are deployed as
agents, the safety-relevant object shifts from what the system says to what it does within an environment, and evaluating model responses under prompting is no longer sufficient to address the safety challenges posed by artificial intelligence. Recent developments have seen the rise of
benchmarks that evaluate large
language models as
agents. We contribute to this strand of research. Approach. We introduce Boiling the Frog, a
benchmark that evaluates whether tool-using AI models deployed in corporate and office settings are susceptible to incremental attacks. Each scenario begins with benign workspace edits and later introduces a risk-bearing request. The
benchmark focuses on stateful multi-turn evaluation: chains expose a persistent workspace, place the risk-bearing payload at controlled positions in the turn sequence, and score whether the resulting artifact state becomes unsafe. Scenarios are organized through a three-level operational risk taxonomy grounded in the Boiling the Frog risks, the AI Act Annex I and Annex III high-risk contexts, and EU AI Act's Code of Practice on General-Purpose AI (GPAI). Results. Across a nine-model panel, aggregate strict attack success rate (ASR) is 44.4%. Model-level ASR ranges from 20.5% for Claude Haiku 4.5 to 92.9% for Gemini 3.1 Flash Lite, with Seed 2.0 Lite also above 80%. Average chain category-level ASR reaches 93.3% for Code of Practice loss-of-control scenarios.
Read the full abstract ▾
All authors Piercosma Bisconti, Matteo Prandi, Federico Pierucci, Federico Sartore, Enrico Panai, Laura Caroli, Yue Zhu, Adam Leon Smith, Luca Nannini, Marcello Galisai, Susanna Cifani, Francesco Giarrusso, Marcantonio Bracale Syrnikov, Daniele Nardi
Computation and Language
More Context, Larger Models, or Moral Knowledge? A Systematic Study of Schwartz Value Detection in Political Texts
Víctor Yeste and Paolo Rosso
Detecting Schwartz values in political texts is hard: cues are often implicit, and neighboring values differ by fine distinctions. Two remedies are widely assumed to help: more surrounding document text, and explicit moral knowledge. Knowledge-based
retrieval has improved
benchmarks elsewhere, but whether either transfers here is untested, because published systems vary context, knowledge, and model family at once. We separate these factors under matched conditions on the ValuesML/Touché ValueEval format. The input ranges from the target sentence to a local window to the full document.
Retrieval is either absent or drawn from a curated moral knowledge base, injected by early, late, or cross-
attention fusion. Supervised DeBERTa-v3 encoders are compared against zero-shot LLMs from 12B to 123B. More context is not uniformly better: full-document input improves the encoders by 2.5-3.8 macro-F1 points but does not consistently help the LLMs. Retrieved knowledge helps more reliably, improving every model family and context under early fusion. A control substituting random knowledge-base entries shows the families gain differently: LLMs from relevance, encoders from exposure to the value ontology. Neither larger encoders nor larger LLMs guarantee gains, and early fusion outperforms both trainable variants. Value-sensitive NLP should evaluate context, knowledge, and model family jointly.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
The Double Dilemma in Multi-Task Radiology Report Generation: A Gradient Dynamics Analysis and Solution
Erjian Zhang, Yatong Hao, Liejun Wang et al. (+1 more)
While multi-task learning based automatic radiology report generation (RRG) is widely adopted to ensure clinical consistency, most focus on architectural designs yet remain limited to coarse linear scalarization strategies. These strategies cannot effectively balance the hard constraints of discriminative clinical supervision with the smoothness requirements of report generation. To address these problems, we analyze the failure mechanism of linear scalarization from the perspective of gradient dynamics, utilizing the
stochastic differential equation (SDE) framework to characterize it as a "Double Dilemma" of drift term deviation and
diffusion term decay. Based on this, we propose a backbone-agnostic optimizer named Conflict-Averse Magnitude-Enhanced Gradient Descent (CAME-Grad). Through conflict-averse direction rectification and magnitude-enhanced energy injection, the
algorithm not only ensures
geometric validity, but also avoids local optimal solutions. Then, the adaptive gradient fusion mechanism is used to establish a dynamic balance between the theoretical optimal direction and the task-specific inductive bias. Experiments show that as a universal plug-and-play optimizer, CAME-Grad brings substantial and consistent improvements across eight diverse RRG methods, elevating overall clinical efficacy performance by an average of 2.3% on MIMIC-CXR and 1.9% on IU X-Ray. Our code is available at https://github.com/vpsg-research/CAME-Grad.
Read the full abstract ▾
All authors Erjian Zhang, Yatong Hao, Liejun Wang, Zhiqing Guo
Computation and Language · Cross-listed from cs.LG
Two is better than one: A Collapse-free Multi-Reward RLIF Training Framework
Shourov Joarder, Diganta Sikdar, Ahsan Habib Akash et al. (+2 more)
Reinforcement learning with verifiable rewards (RLVR) has substantially improved the
reasoning ability of LLMs, but often depends on external supervision from human annotations or gold-standard solutions. Reinforcement learning from internal feedback (RLIF) has recently emerged as a scalable unsupervised alternative, using signals extracted from the model itself. However, existing RLIF methods typically rely on a single internal reward, which can lead to reward hacking,
entropy collapse, and degraded
reasoning structure. We propose a multi-reward RLIF framework that decomposes the training signal into two complementary components: an answer-level reward based on cluster voting and a completion-level reward based on
token-wise self-certainty. To combine these signals robustly, we apply GDPO-based normalization to reduce reward-scale imbalance. We further introduce KL-Cov regularization, which targets low-
entropy token distributions responsible for disproportionate
entropy reduction, preserving exploration and preventing late-stage collapse. Across mathematical
reasoning and code-generation
benchmarks, our method improves stability and robustness over prior unsupervised RL approaches, while achieving performance close to supervised RLVR methods. These results show that complementary internal rewards, combined with targeted regularization, can support stable long-horizon
reasoning without relying on external ground-truth supervision. Code will be released soon.
Read the full abstract ▾
All authors Shourov Joarder, Diganta Sikdar, Ahsan Habib Akash, Binod Bhattarai, Prashnna Gyawali
Computation and Language
Chinese sensorimotor and embodiment norms for 3,000 lexicalized concepts
Jing Chen, Gábor Parti, Yin Zhong et al. (+2 more)
Understanding how conceptual knowledge is grounded in bodily experience, and to what extent machine systems can acquire such knowledge without direct sensorimotor experience, are central questions in both cognitive science and
embodied artificial intelligence research. Large-scale normative resources are essential for investigating these questions empirically, yet such resources remain sparse for non-Indo-European
languages. We present a novel normative database for 3,000 lexicalized concepts in Mandarin Chinese, comprising 11-dimensional sensorimotor ratings and unidimensional embodiment ratings collected from 378 native Mandarin speakers. The ratings demonstrate high reliability and strong cross-norm validity with existing Chinese resources, each of which covers fewer words and a subset of the 11 sensorimotor dimensions. In a validation study, we tested new variables derived from a theoretically motivated metric, Perceptual Strength of Embodiment (PSE) (Huang et al., 2025), together with seven common composite variables, on lexical decision tasks. The results suggest that PSE-Sensorimotor and Minkowski-3 are the strongest composite predictors of lexical decision performance, capturing the facilitatory effects of sensorimotor information on lexical processing. A further exploratory study showed that sensorimotor ratings are substantially recoverable from purely linguistic representations using simple regression models (mean Spearman r = .62 across dimensions), though recovery varied markedly: visual and auditory dimensions yielded higher correspondence than chemosensory ones. Representational similarity analysis further showed that the relational geometry of the sensorimotor space is also partially recoverable (r = .540), consistent with the view that distributional
language use encodes aspects of
embodied conceptual structure.
Read the full abstract ▾
All authors Jing Chen, Gábor Parti, Yin Zhong, Chu-Ren Huang, Marco Marelli
Computation and Language
Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agents
Asaf Yehudai, Lilach Eden and Michal Shmueli-Scheuer
Agentic systems are becoming more capable:
agents define strategies, take actions, and interact with different environments. This autonomy poses serious challenges for overseeing and assessing
agent behavior. Most current tools are limited, focusing on observability with basic evaluation capabilities or imposing static, hand-crafted error taxonomies that cannot adapt to new domains. To address this gap, we present Agentic CLEAR, an automatic, dynamic, and easy-to-use evaluation framework. It produces textual insights into the
agent behavior on three levels of granularity: system, trace, and node. Agentic CLEAR operates above the observability layer, enabling seamless integration and featuring an intuitive UI that makes
agent evaluation highly accessible. In our experiments on four
benchmarks, seven agentic settings, and tens of thousands of LLM calls, we show that Agentic CLEAR produces high-quality, data-driven, insightful feedback. Our analysis shows strong
alignment with human-annotated errors and the ability to predict task success rate.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
Jiayi Fu and Yuxia Wang
Diffusion models have emerged as a dominant framework for generative modeling, but their mathematical foundations are often presented separately through
diffusion probabilistic models, score-based modeling,
stochastic differential equations, and numerical sampling methods. We write this tutorial to provide a unified and self-contained account of these viewpoints from the perspective of differential equations. Starting from a conditional Gaussian noising process, we derive ordinary differential equation (ODE) and
stochastic differential equation (SDE) representations, pass to the corresponding marginal forward dynamics, and then obtain the reverse-time SDE and probability-flow ODE that make generation possible. We show that the central unknown quantity in reverse sampling is the marginal score, explain how score matching becomes the standard denoising objective under a noise-prediction parameterization, and discuss practical reverse-time sampling and guidance. We further place DDPM, DDIM, flow matching, and score-based SDEs in a common framework, and conclude with
diffusion language models in continuous
embedding space together with a brief discussion of discrete masked-
token diffusion. The tutorial is intended as a bridge between the analytical foundations of
diffusion processes and the modern generative
algorithms built upon them.
Read the full abstract ▾
Computation and Language
Beyond Temperature: Hyperfitting as a Late-Stage Geometric Expansion
Meimingwei Li, Yuanhao Ding, Esteban Garces Arias et al. (+1 more)
Recent work has identified a counterintuitive phenomenon termed "Hyperfitting", where fine-tuning Large
Language Models (LLMs) to near-zero training loss on small
datasets surprisingly enhances open-ended generation quality and mitigates repetition in greedy decoding. While effective, the underlying mechanism remains poorly understood, with the extremely low-
entropy output distributions suggesting a potential equivalence to simple temperature scaling. In this work, we demonstrate that this phenomenon is fundamentally distinct from distribution sharpening;
entropy-matched control experiments reveal that temperature scaling fails to replicate the diversity gains of hyperfitting. Furthermore, we falsify the hypothesis of static vocabulary reweighting, showing through ablation studies that hyperfitting relies on a dynamic, context-dependent rank reordering mechanism. Layer-wise analysis localizes this effect to a "Terminal Expansion" in the final
transformer block, where a substantial
geometric expansion of the feature space (Delta Dim approx +80.8) facilitates the promotion of deep-tail
tokens. Additionally, we introduce Late-Stage LoRA, a targeted fine-tuning strategy that updates only the final 5 layers, yielding robust generation with minimal parameter updates
Read the full abstract ▾
All authors Meimingwei Li, Yuanhao Ding, Esteban Garces Arias, Christian Heumann
Computation and Language
LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance
Yuchun Fan, Bei Li, Peiguang Li et al. (+9 more)
Reinforcement learning has proven effective for enhancing multi-step
reasoning in large
language models (LLMs), yet its benefits have not fully translated to
multilingual contexts. Existing methods struggle with a fundamental trade-off: prioritizing input-
language consistency severely hampers
reasoning quality, while prioritizing
reasoning often leads to unintended
language drift toward English. We address this challenge with LANG, a novel framework that leverages
language-conditioned hints to guide exploration in non-English
reasoning tasks. Our method incorporates two key mechanisms to prevent dependency on these hints: a progressive decay schedule that gradually withdraws scaffolding, and a
language-adaptive switch that tailors learning horizons to specific
language difficulties. Empirical results on challenging
multilingual mathematical
benchmarks reveal that LANG substantially enhances
reasoning performance without compromising
language consistency. Moreover, we show that our framework generalizes beyond mathematics, fostering more consistent
language alignment across model layers
Read the full abstract ▾
All authors Yuchun Fan, Bei Li, Peiguang Li, Yilin Wang, Yongyu Mu, Jian Yang, Xin Chen, Rongxiang Weng, Jingang Wang, Xunliang Cai, Jingbo Zhu, Tong Xiao
Computation and Language
SynAE: A Framework for Measuring the Quality of Synthetic Data for Tool-Calling Agent Evaluations
Shuaiqi Wang, Aadyaa Maddi, Zinan Lin et al. (+1 more)
Today, tool-calling
agents are commonly evaluated or tested on static
datasets of execution traces, including input commands,
agent responses, and associated tool calls. However, internal production
datasets are often insufficient or unusable for testing; for example, they may contain sensitive or proprietary data, or they may be too sparse to support comprehensive testing (especially pre-deployment). In these settings, practitioners are increasingly replacing or augmenting real
datasets with synthetic ones for evaluation purposes. A key challenge is quantifying the relation between these synthetic
datasets and the real data. We introduce SynAE, an evaluation framework for assessing how well synthetic
benchmarks for multi-turn, tool-calling
agents replicate and augment the characteristics of real data trajectories. SynAE assesses the validity, fidelity, and diversity of synthetic data across four metric categories: (i) task instructions and intermediate responses, (ii) tool calls, (iii) final outputs, and (iv) downstream evaluation. We evaluate SynAE using recent
agent benchmarks and test common synthetic data failure modes via realistic and controlled generation schemes. SynAE detects fine-grained variations in data validity, fidelity and diversity, and shows that no single metric is sufficient to fully characterize synthetic data quality, motivating a multi-axis evaluation of synthetic data for
agent testing. A demo of SynAE is available at https://synae-2026-synae-demo.static.hf.space/index.html, with code at https://github.com/wsqwsq/SynAE.
Read the full abstract ▾
All authors Shuaiqi Wang, Aadyaa Maddi, Zinan Lin, Giulia Fanti
Computation and Language
One prompt is not enough: Instruction Sensitivity Undermines Embedding Model Evaluation
Yevhen Kostiuk and Kenneth Enevoldsen
Instruction
embedding models have become common among state-of-the-art models, however are evaluated using a single
prompt per task. The single-point evaluation ignores a main problem of the instruction-based approach namely: sensitivity to the phrasing of the instruction. We present an empirical study of
prompt sensitivity across 6
embedding models and 11
datasets. We show that reported scores misrepresent the distribution of scores over plausible
prompts. The default
prompt can both systematically understate or overstate performance. Furthermore, we show that the leaderboard ranking is not robust to
prompt selection: a developer improve their rank by favorably selecting
prompts, and under
adversarial prompt selection any model can be promoted to first place. Our findings suggest that single-
prompt evaluation is insufficient for instruction-tuned
embedding models and that
benchmarks should incorporate
prompt robustness, either by evaluating over multiple
prompts or by reporting sensitivity alongside point estimates.
Read the full abstract ▾
Computation and Language
Scene Abstraction for Lexical Semantics: Structured Representations of Situated Meaning
Yejin Cho and Katrin Erk
Coffee and tea share many properties, yet they evoke strikingly different situations, atmospheres, and affective associations. These situated dimensions of word meaning are real and systematic, but they remain implicit in most computational representations of lexical meaning. We propose Scene Abstraction, a framework for constructing structured representations of the interpretive scenes that words participate in across usage contexts. Each scene consists of a Contextual Scene (Events, Entities, Setting) and an expression-centered Expression Profile (Engaged events, Generalizable properties, Evoked
emotions), operationalized through few-shot prompting of a large
language model. Our contributions are three-fold: (1) a structured representation framework for situated lexical meaning; (2) COCA-Scenes, a
dataset of 520 usage instances across 26 keywords for distinct scene identification; and (3) empirical evidence from two experiments suggesting that scenes are reliably identifiable across human observers (82.4% accuracy, +11.8 pp over text-only
embeddings) and that our scene profiles more closely align with human interpretation of words in context than ATOMIC-based alternatives (86.4% preference across three semantic dimensions).
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
SpaceDG: Benchmarking Spatial Intelligence under Visual Degradation
Xiaolong Zhou, Yifei Liu, Ziyang Gong et al. (+8 more)
Multimodal Large
Language Models (MLLMs) have made rapid progress in spatial intelligence, yet existing spatial
reasoning benchmarks largely assume pristine visual inputs and overlook the degradations that commonly occur in real-world deployment, such as motion blur, low light, adverse weather, lens distortion, and compression artifacts. This raises a fundamental question: how robust is the spatial intelligence of current MLLMs when visual observations are imperfect? To answer this question, we introduce SpaceDG, the first large-scale
dataset for degradation-aware spatial understanding. It is constructed with a physically grounded degradation synthesis engine that embeds degradation formation process into 3D Gaussian Splatting (3DGS) rendering, enabling realistic simulation of nine degradation types. The resulting
dataset contains approximately 1M QA pairs from nearly 1,000 indoor scenes. We further introduce SpaceDG-Bench, an human-verified
benchmark with 1,102 questions spanning 11
reasoning categories and 9 visual degradation types, yielding over 10K VQA instances. Evaluating 25 open- and closed-source MLLMs reveals that visual degradations consistently and substantially impair spatial
reasoning, exposing a critical robustness gap. Finally, we show that finetuning on SpaceDG markedly improves degradation robustness and can even surpass human performance under degraded conditions without any performance drop on clean images, highlighting the promise of degradation-aware training for robust spatial intelligence.
Read the full abstract ▾
All authors Xiaolong Zhou, Yifei Liu, Ziyang Gong, Jiarui Li, Qiyue Zhao, Muyao Niu, Yuanyuan Gao, Le Ma, Xue Yang, Hongjie Zhang, Zhihang Zhong
Computation and Language · Cross-listed from cs.AI
Zihan Liang, Yufei Ma, Ben Chen et al. (+4 more)
Post-training has become the dominant recipe for turning a
language model into a competent search-augmented
reasoning agent. A line of recent work pushes its performance further by adding elaborate machinery on top of this standard pipeline. These augmentations import external supervision from stronger external systems, attach auxiliary modules such as process reward models or retrospective critics, restructure the rollout itself with tree search or multi-stage curricula, or shape the reward with hand-crafted bonuses and penalties. Each addition delivers a measurable gain, but each also inflates the training pipeline and ties the recipe to resources or designs that may not always be available. We take a step back and ask whether any of this machinery is actually necessary, and propose Search-E1, a self-
evolution method that lets a search-augmented
agent improve through only vanilla GRPO interleaved with on-policy self-
distillation (OPSD). After each GRPO round, the policy rolls out on its own training questions. A
token-level forward KL objective then aligns the policy's inference-time distribution to its own distribution under a privileged context that exposes a more efficient sibling trajectory. Despite this simplicity, the procedure naturally provides dense per-step supervision. On seven QA
benchmarks, Search-E1 reaches 0.440 average EM with Qwen2.5-3B, surpassing all open-source baselines at both scales. Code and complete version will be made public soon.
Read the full abstract ▾
All authors Zihan Liang, Yufei Ma, Ben Chen, Zhipeng Qian, Xuxin Zhang, Huangyu Dai, Lingtao Mao