Computation and Language · Cross-listed from cs.LG
Weightless Fine-Tuning: Personalizing LLMs via Logit-Space Transport
Bohan Zhang, Anqi Ni, Yixin Wang et al. (+1 more)
Supervised fine-tuning (SFT) is a standard approach for adapting LLMs to a target distribution, but in settings such as personalization, where each author requires separate weight access, optimization, storage, and retraining, its costs become prohibitive. We propose Weightless Fine-Tuning (WFT), a training-free decoding-time method that approximates the distributional effect of SFT without weight updates. WFT computes supervised residuals on an author's training sequence and transports them to the current
prompt through a cross-prefix transport operator estimated from dropout-induced cross-covariance. The operator captures how a perturbation at one context propagates to predictions at another, replacing gradient-based parameter updates with logit-space corrections. On three LaMP personalization
benchmarks, WFT achieves the best average performance across
datasets, matches or exceeds SFT on individual tasks, and outperforms other lightweight baselines on average. In a budget-controlled comparison, WFT approaches SFT performance using less than 7% of the effective computation. Logit-level analysis shows a cosine similarity of 0.875 between the logit shifts induced by WFT and SFT over 95% of the next-
token probability mass, suggesting that WFT captures the distributional effect of supervised adaptation without modifying model weights.
Read the full abstract ▾
All authors Bohan Zhang, Anqi Ni, Yixin Wang, Paramveer S. Dhillon
Computation and Language
Better, Faster, Stronger: Programmatic Skill Learning Best Reduces Agent Cost
Zixi Huang, Xiheng Wang, Andrew Wang et al. (+4 more)
Recently, the practice of augmenting LLM
agent capability with skills has gained prevalence. We explore the cost effective adaptation of
agents to novel domains by means of learning skills. Existing works focus on performance gain over cost effectiveness. As a result, little is known about what skill learning strategies save cost. We argue that among all the different skill learning methods, those that view skills as programs can achieve the best cost reduction. By executing sequences of actions deterministically, a program-augmented
agent can reliably and cheaply achieve goals that would otherwise require trial and error and risk degenerate behavior over long horizons. An
agent can learn at inference time by incrementally discovering these programs and equipping them for future tasks. We hypothesize that past trajectories contain enough signal to guide skill learning, even without replay or validation, provided the
agent can learn to analyze them. To test our claims, we propose SpeedRunner, a coding
agent that analyzes trajectories and refactors skills for better performance on future tasks. Across three different
embodied environments, we show that SpeedRunner consistently achieves the frontier in learning and cost reduction while remaining robust against distribution shifts and environmental randomness.
Read the full abstract ▾
All authors Zixi Huang, Xiheng Wang, Andrew Wang, William Jurayj, Bernal Jiménez Gutiérrez, Daniel Khashabi, Nicholas Andrews
Computation and Language
Gloss-Free Representation Learning for Cross-Dataset Sign Spotting
Oğuz Akif Tüfekcioğlu, Ezgi Ekin, Mustafa Kaan Çevik et al. (+1 more)
Sign-
language research for resource-constrained
languages is often limited by the cost of dense linguistic labels such as glosses, temporal boundaries, and sign order. Broadcast news offers a practical alternative by pairing continuous signing with spoken-
language transcripts, but this supervision is weak since text and signing are loosely aligned. Morphologically rich
languages such as Turkish add further difficulty, as the same lexical meaning can appear in many inflected forms while some derived forms should remain distinct. We study whether weak transcript-based supervision can pretrain a reusable sign encoder in this setting, where poor text normalization can fragment pseudo-gloss targets and weaken representation learning. Unlike prior pseudo-gloss pipelines designed mainly to improve translation, we test whether the pretrained encoder transfers as a reusable representation for cross-
dataset sign spotting. We pretrain on TSL-News, a new Turkish broadcast corpus, using pseudo-gloss labels derived from transcripts rather than manual annotation, comparing rule-based morphological lemmatization with constrained LLM-assisted normalization over a fixed vocabulary. We evaluate the learned representations via cross-
dataset sign spotting on a new TSL Spotting
Benchmark built from the TSL Dictionary corpus. The LLM-assisted encoder raises top-5 temporal localization mean IoU from 0.235 to 0.465, with 56.2% of examples reaching an IoU of at least 0.50; a frequency analysis suggests this gain is not mainly driven by memorizing frequent pseudo-gloss labels. In a downstream translation check, the same pretraining improves BLEU-4 from 9.60 to 11.04 and ROUGE from 23.48 to 27.43. These results show that loosely aligned broadcast data can provide effective weak supervision for learning sign representations that capture both lexical content and temporal structure.
Read the full abstract ▾
All authors Oğuz Akif Tüfekcioğlu, Ezgi Ekin, Mustafa Kaan Çevik, Hacer Yalim Keles
Computation and Language
ConVAWG: A Retrieval-Grounded Framework for Controlled Synthetic Dialogue Generation in Violence Against Women and Girls
Chen Lyu, Xingwei Tan, Simon Cullen et al. (+4 more)
Synthetic dialogue generation offers a way to study conversational dynamics in sensitive domains where real data are difficult to access, release, or annotate. The underlying abuse may occur online or offline: threats and coercion can appear directly in messages, while behaviours such as surveillance, isolation, stalking, and physical violence may be planned, disclosed, or referred to conversationally.
Privacy and legal constraints make it difficult the release of large-scale real conversation
datasets; existing work has mostly focused on sentence-level toxicity of online abuses, leaving a gap in modelling abuse as a relational and temporally unfolding phenomenon. In this work, we focus on modelling Violence Against Women and Girls (VAWG) scenarios as multi-turn dialogues. We introduce ConVAWG, a
retrieval-grounded framework for generating CPS-aligned synthetic VAWG chat dialogues. ConVAWG builds scenarios from persona seeds, demographic patterns reported by the UK Office for National Statistics, official crime definitions, and retrieved Domestic Homicide Review cases; converts them into hierarchical event timelines; generates multi-scene role-play dialogues; and applies targeted activation-steered toxicity control to appropriate utterances. We release over 6,000 multi-turn dialogue events across 200 scenarios with rich scenario-, event-, and turn-level metadata. Extensive human evaluation, LLM-as-Judge assessment, ablations, and downstream tasks show strong dialogue quality and domain fidelity.
Read the full abstract ▾
All authors Chen Lyu, Xingwei Tan, Simon Cullen, Shelley Wilson, Lois Arthurs, Arshad Jhumka, Gabriele Pergola
Computation and Language · Cross-listed from cs.LG
Beyond a Bag of Features: Set-Level Instability in Sparse Autoencoders
Nikolai Bolik, Lennart Stöpler and Artur Andrzejak
Shani et al. (2026) show that LLM representations broadly recover human category boundaries, while failing to reflect fine-grained typicality structure. Their analysis uses cosine similarity over dense model representations. We revisit their approach using overlap over active sparse autoencoder (SAE) latent sets as a more interpretable similarity measure. We first verify that this set-level measure is meaningful: SAE latent sets can recover union-like compositional structure in controlled toy models and induce semantically coherent neighborhoods in natural text. Extending the human-concepts analysis to SAE set similarities, we find that SAE activation sets do not recover human category boundaries or within-category typicality more faithfully than dense
embeddings or residual-stream states, but instead track model-internal similarity structure. To probe this gap further, we study active latent sets under well-controlled semantic modifications, revealing a substantial mismatch between human judgements of conceptual change and change in the SAE active set. We interpret this as evidence that, outside idealised settings, SAE features do not compose via simple bag-of-features semantics.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation
Shiyu Xuan and Zechao Li
GUI Visual Grounding is a fundamental capability for GUI
agents. Existing models typically freeze their parameters after deployment, limiting their ability to adapt to unseen interfaces. Although recent methods attempt to adapt models via test-time reinforcement learning, they cannot reflect upon failed exploration. To overcome this, we propose a Test-Time Self-Evolving framework that enables models to improve after deployment without human-annotated ground truth. It constructs a closed-loop of Exploration, Evaluation, Reflection, and Internalization. Specifically, the
agent first explores unseen interfaces by predicting grounding coordinates for given instructions. To evaluate these explorations, we introduce an MLLM-based Reflector to assess the generated results and provide the corresponding
reasoning reflections. To internalize reflection knowledge into the model weights, we propose Reflection-Guided On-Policy Self-
Distillation, which translates high-level
reasoning into dense
token-level supervision via a conditioned self-teacher. Furthermore, we design a Contrastive Calibration method to prevent incorrect auto-regressive prefixes from corrupting the supervisory signals during failed explorations. Extensive experiments across six
benchmarks demonstrate our framework's effectiveness, achieving an average accuracy improvement of 7.4% over the base model. To the best of our knowledge, this is the first work to successfully exploit on-policy self-
distillation for test-time adaptation in GUI visual grounding. By filling the gap in post-deployment adaptation, our framework completes the self-evolving capability of GUI
agents. The code will be released.
Read the full abstract ▾
Computation and Language
From Interpretability to Control: Insights from Six Years of the TrustNLP Workshop
Rahul Gupta, Abhinav Mohanty, Anaelia Ovalle et al. (+10 more)
The Workshop on Trustworthy Natural
Language Processing (TrustNLP), co-located with major ACL conferences since 2021, has grown from 8 proceedings papers to 41 over six editions, documenting a field-wide transition from post-hoc interpretability of static models to mechanistic understanding and proactive control of generative systems. We synthesize insights from all 144 proceedings papers, classifying them along six trust dimensions grounded in established frameworks (TrustLLM, DecodingTrust). We observe co-occurrences with capability emergence. The release of the first high-impact chat models activated all trust dimensions simultaneously, while subsequent model generations shifted focus toward truthfulness and safety
alignment. Analysis from the classification study reveals that truthfulness is the fastest-growing dimension (absent in 2021-2022, comprising 37% of papers by 2025-2026),
fairness remains the most consistent theme, and explainability exhibits a U-shaped trajectory; declining as post-hoc methods lost relevance but resurging in 2026 through mechanistic interpretability. A cross-venue comparison with ACL, NAACL, EACL, and EMNLP (~2K papers) in the same period shows that TrustNLP's topical distribution closely follows the field average. We identify four structural insights and conclude with actionable directions for the research community.
Read the full abstract ▾
All authors Rahul Gupta, Abhinav Mohanty, Anaelia Ovalle, Anil Ramakrishna, Anubrata Das, Apurv Verma, Jwala Dhamala, Ninareh Mehrabi, Tharindu Kumarage, Yada Pruksachatkun, Yang Trista Cao, Kai-Wei Chang, Aram Galstyan
Computation and Language · Cross-listed from cs.CV
MultiModal Code-Switching: Interleaving Visual Objects into Language for Explicit Object-Level Alignment
Changhao Xiang, Shangyu Xing, Zhen Wu et al. (+2 more)
Existing
Multimodal Large
Language Models (MLLMs) predominantly rely on image-text pairs for modality
alignment pretraining, mapping global image representations to long textual descriptions. However, this image-level
alignment suffers from referential ambiguity: models struggle to infer the correspondences between multiple visual objects and textual entities from the global representation, leading to data inefficiency and suboptimal semantic grounding. To address this, we propose
MultiModal Code-Switching (MMCS), a novel pretraining paradigm that provides explicit object-level supervision. Inspired by the linguistic phenomenon of code-switching, MMCS interleaves
vision and
language by replacing textual entities with their corresponding visual objects, enforcing local
vision-
language grounding. We further develop a scalable data synthesis pipeline to generate a pretraining
dataset of 773K samples with accurate object-entity correspondences. Experiments show that MMCS is highly data-efficient: with only 50K samples, it matches or surpasses models trained on 600K image-text pairs. Furthermore, MMCS consistently improves visual grounding and perception capabilities across varying model scales.
Read the full abstract ▾
All authors Changhao Xiang, Shangyu Xing, Zhen Wu, Jianbing Zhang, Xinyu Dai
Computation and Language
The Illusion of Cross-Lingual Safety in Low-Resource Languages
Abigail Oppong, P Sam Sahil, Tadesse Destaw Belay et al. (+12 more)
Safety
alignment in large
language models (LLMs) is largely developed in English, assuming these safeguards generalize across
multilingual settings. However, this assumption remains underexplored and exposes a vulnerability in low-resource
languages. We investigate cross-lingual safety transfer in four African
languages, Twi, Hausa, Amharic, and Swahili, using LoDNA, a new safety
dataset that pairs literal translations with culturally localized
prompts. To move beyond generation-based evaluation, we propose a latent
geometric framework that probes hidden-state refusal representations in LLMs. Our experimental results show that cross-lingual safety transfer is severely limited; harmful
prompts retain less than 10% of the English refusal signal across most
language-model pairs. Literal and localized
prompts are semantically aligned (cosine 0.95-0.996) but drift across layers, suggesting models encode the concepts without routing them to safety mechanisms. These findings demonstrate that current
multilingual safety
alignment is superficial, providing strong evidence against the assumption of a universal,
language-agnostic harm
manifold within the specific low-resource
languages studied. Warning: This paper contains example data that may be offensive or harmful.
Read the full abstract ▾
All authors Abigail Oppong, P Sam Sahil, Tadesse Destaw Belay, Maryam Ibrahim Mukhtar, Esmael Ahmed Abdu, Tassallah Abdullahi, Jessica Oparebea, Saminu Mohammad Aliyu, Idris Abdulmumin, Abubakar Juma Chilala, Nicholaus Dismas Ladislaus, Alfred Malengo Kondoro, Lemofouet Valdini Douglace, Shamsuddeen Hassan Muhammad, Seid Muhie Yimam
Computation and Language
Attention-Path Fragility as an Uncertainty Signal in Large Language Models
Minsoo Kim, Sungyoung Ji, Kisung Moon et al. (+1 more)
We propose that a model's uncertainty about a
token is reflected not only in the breadth of its output distribution but also in whether a confident prediction is fragile under perturbation of its
attention pathways. We instantiate this as ASMI (
Attention-Subnetwork Mutual Information), a training-free
estimator that masks
attention heads and measures the BALD mutual information among the resulting subnetworks, with a semantic-agreement kernel to discount surface-form disagreement. The signal is not a restatement of output confidence: on grounded QA an out-of-fold test shows it adds error-predictive information beyond single-pass confidence and
entropy, concentrated in confident-but-fragile predictions, where acting on it roughly halves the retained error of a confidence filter. The distinctness is regime-graded, so ASMI predicts its own domain of applicability, strong where answers are routed through provided context and bounded by design where they are recalled from parametric knowledge. Sem-ASMI reads the signal from a single greedy response, without the
stochastic generations the strongest baselines require, and ties or beats Semantic
Entropy on ten of the twelve grounded
benchmark-backbone settings. Across the same twelve settings, the best ASMI variant, typically the adaptive one reusing the ten samples already drawn for the baselines, ties or leads the strongest baseline in eight, significantly in three under a paired test. On parametric QA all variants revert to or below the zero-cost MSP baseline, exactly as predicted, and the estimates are near-deterministic across reruns. A head-level analysis shows that what tracks this boundary is not the presence of head-level fragility but whether that fragility couples to errors.
Read the full abstract ▾
All authors Minsoo Kim, Sungyoung Ji, Kisung Moon, Ilyong Yoon
Computation and Language
Actions Speak Louder than Words: Measuring Cross-Lingual Policy Retention in Tool-Using Agents
Sourabrata Mukherjee, Kalika Bali and Sunayana Sitaram
When a tool-using
agent is given the same task in a different
language, does it still take the same steps?
Multilingual evaluation rarely asks: it compares final answers and discards the actions. Yet those actions are the product: they fix cost and latency, decide how the system fails, and are the only auditable part of its behaviour. We make the action policy the measured object across 8 models, 6 parallel
benchmarks and 41
languages (2.38M rollouts). The naive measurement fails: five confounds sit between raw trace similarity and any defensible claim, each able to flip a conclusion. Short traces score higher, empty traces score perfectly, unrelated traces agree by chance over half the time, the gap is capped by each model's reproducibility, and a model asked the same question twice in one
language answers differently, leaving no baseline. We remove all five, and every correction makes the effect larger. Divergence proves structural, not sampling noise: it survives greedy decoding in every
cell and stays flat as temperature rises, even as models grow less self-consistent. Normalised by their own reproducibility, four very different frontier models converge under greedy decoding, each keeping 71-73% of its action policy across
languages, with model identity explaining only 5.7% of the variance. Below roughly 10B parameters it breaks down, and the ordering among smaller models is largely an artifact of a chance floor we measure by permutation rather than assume.
Agents route non-English tasks through English; this pivot is causally load-bearing, confirmed by a pre-registered prediction across four models, and models will not abandon it when told to. Finally, a single trace-extraction regex, not the model, manufactured a
multilingual failure: two worked examples raise one model's measured accuracy twenty-sixfold while its accuracy on readable outputs barely moves.
Read the full abstract ▾
Computation and Language
Multiclass Sentiment Analysis for Identifying Political Viewpoints
Girma Yohannis Bade, Olga Kolesnikova, Jose Luis Oropeza et al. (+1 more)
The rapid growth of social media has created vast amounts of political discourse, which provides valuable opportunities to analyze public opinions and identify different political perspectives. Sentiment Analysis (SA) is a core task in Natural
Language Processing (NLP) that allows the computational study of attitudes and opinions in textual data, and has become increasingly important for understanding political discourse. In this work, we investigate multiclass sentiment analysis of political view- points on social media, that is to automatically discriminate multiple sentiment classes over political issues and figures. To solve this task we design and evaluate two machine-learning approaches based on XGBoost and BERT. We train and evaluate the models on a labeled
dataset of political social media posts using standard classification metrics. The experimental results show that the XGBoost model reaches an F1-score of 0.2835 and the BERT- based model reaches an F1-score of 0.2806 on the test set. These results demonstrate the challenge of classifying complex and contextualized political discourse sentiment and provide a baseline for future research in multiclass political sentiment analysis.
Read the full abstract ▾
All authors Girma Yohannis Bade, Olga Kolesnikova, Jose Luis Oropeza, Grigori Sidorov
Computation and Language · Cross-listed from cs.LG
ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantization
He-Yen Hsieh and H. T. Kung
ReRound (Reconstructive Rounding) is a post-training quantization method that addresses the midpoint ambiguity inherent in standard round-to-nearest (RTN) schemes when quantizing weights near the centers of quantization intervals. Starting from a pretrained LLM, ReRound trains a conditional
diffusion model to produce continuous reconstructions of low-bit weights for the LLM. These reconstructed weights act as a guidance signal to disambiguate the rounding direction of weights located close to interval midpoints. To integrate this reconstruction-guided rounding with conventional RTN, ReRound introduces a tolerance metric measuring how far the quantized weight (not the final quantized integer) is away from the midpoint: quantized weights within a tolerance region around midpoints are quantized using
diffusion-based reconstructions, whereas weights closer to quantization boundaries are quantized with RTN. By sweeping the tolerance parameter, ReRound generates multiple candidate quantized integer weight
matrices and selects the de-quantized weight
matrix candidate whose leading singular values most closely match those of the original full-precision weights. This selected candidate determines the tolerance parameter ReRound uses. ReRound is particularly effective for smaller LLMs. Across a range of such models, it consistently outperforms standard RTN for 3-bit and 4-bit weight quantization. ReRound achieves superior accuracy compared to an extensive set of calibration-free methods, remains competitive with calibration-dependent approaches, and operates entirely offline, introducing no additional overhead during low-bit inference. The ReRound strategy represents a new approach for low-bit quantization. The method applies to AI models beyond LLMs. This paper focuses on its applications to small LLMs.
Read the full abstract ▾
Computation and Language
TEAMMix: Taxonomy Enrichment Augmentation and Minority-augmented Mixing Strategy for LLM-enhanced Weak-Supervised Hierarchical Text Classification
Jian Zhang, Zhuohao Yang, Songlin Lei et al. (+6 more)
Hierarchical Text Classification (HTC), as a critical text mining task, faces challenges such as complex label hierarchies and class imbalance. Existing methods based on large
language models (LLMs) struggle to be efficiently applied to this task due to issues like lengthy
prompts and loss of label structural information. To address these limitations, this paper proposes a weakly supervised HTC framework enhanced by LLM-based data augmentation. The framework first enriches the label hierarchy semantically through keyword generation and corpus mining, thereby enhancing the model's understanding of labels. Subsequently, it guides the LLM to generate pseudo-samples to mitigate the long-tail problem, and employs a Gaussian mixture model for confidence-based resampling to optimize the quality of generated data. Experimental results demonstrate that the proposed method effectively improves the reliability of LLM-generated pseudo-labels and significantly enhances classification performance on fine-grained and imbalanced
datasets.
Read the full abstract ▾
All authors Jian Zhang, Zhuohao Yang, Songlin Lei, Bangli Liu, Ziwei Wang, Xufeng Weng, Gehan Amaratunga, Yu Lin, Hongwei Wang
Computation and Language
myMediWhisper: Construction of Burmese Medical Speech Corpus and Whisper Fine-Tuning for Clinical Dialogue ASR
Ye Kyaw Thu, Ye Bhone Lin, Thura Aung et al. (+6 more)
Although Whisper models benefit from large-scale
multilingual pre-training, their performance on Burmese medical
speech remains limited. This work presents a Burmese medical
speech recognition framework built on a high-quality 28-hour corpus recorded and validated by native speakers. We fine-tune Whisper models using full fine-tuning (FFT) and parameter-efficient fine-tuning (PEFT) with LoRA. To evaluate robustness, we apply waveform- and spectrogram-level data augmentation under controlled noise and simulated room acoustics. While augmentation reduces performance on clean
speech, it significantly improves robustness in noisy and reverberant environments across FFT and PEFT settings. Our best-performing system, fully fine-tuned myMediWhisper-Medium without augmentation, achieves a state-of-the-art Word Error Rate (WER) of 23.44%, outperforming much larger general-domain fine-tuned models.
Dataset and other resources can be found at the Huggingface repository: https://huggingface.co/
datasets/LULab/mediTalk-mm-rdy.
Read the full abstract ▾
All authors Ye Kyaw Thu, Ye Bhone Lin, Thura Aung, Htet Arkar, Myat Oo Swe, Thet Htet San, Min Thiha Tun, Thazin Myint Oo, Thepchai Supnithi
Computation and Language · Cross-listed from cs.IR
Self-Knowledge Retrieval Augmented Generation Framework for Patent Matching
Jian Zhang, Songlin Lei, Zhuohao Yang et al. (+6 more)
Patent
retrieval and matching based on large
language models (LLMs) play a vital role in intellectual property protection. However, due to the complex structure of patent documents, dense technical terminology, and multi-modal information, traditional methods struggle to accurately identify subtle differences between patents. Existing LLM-based patent matching approaches typically rely on domain-specific pretrained or instruction tuning, which often entail high manual labeling costs and catastrophic forgetting. While
retrieval-augmented generation (RAG) methods introduce external knowledge they fail to fully leverage LLM's capability to automatically parse patents and mine deep semantic relationships. To address these limitations, this paper proposes a self-knowledge RAG framework that guides LLMs to autonomously extract key technical entities and construct hierarchical ontological structures from patent matching queries, thereby enabling query expansion and precise
retrieval. The method integrates the FAISS
retrieval with a generative matching mechanism, leveraging self-knowledge to enhance the model's understanding of patent innovations and significantly improve
retrieval and matching accuracy. Experimental results demonstrate the outstanding performance of the proposed method on real-world patent
datasets, validating its effectiveness and application potential.
Read the full abstract ▾
All authors Jian Zhang, Songlin Lei, Zhuohao Yang, Bangli Liu, Ziwei Wang, Xufeng Weng, Gehan Amaratunga, Yu Lin, Hongwei Wang
Computation and Language · Cross-listed from cs.LG
Mapping and Measuring the Behavioral Evolution of Large Language Models
Dong Qiao, Chris Ding and Jicong Fan
Benchmark leaderboards summarize how well a
language model performs, but not how its behavior relates to that of other models or changes across generations. We characterize the output behavior of 32 models from six families using their responses to a shared bank of 10{,}000
prompts. After
embedding each response, we construct three complementary sentence-level dissimilarities: an aligned mean per-
prompt distance, which is a pseudometric on observed model responses; a PCA-compressed summary of
prompt-wise disagreement; and an
alignment-free Gromov–Wasserstein discrepancy between models' internal response geometries. We use these constructions to study static organization and temporal change on a release-date axis through behavioral maps, family-wise drift, hierarchical clustering, cross-family convergence, and response-cloud dispersion. Across the three constructions, model families form coherent clusters, with gpt-2 as a global outlier; cross-family distances decrease over time; and several recent
reasoning-oriented models have comparatively compact response clouds. A
token-level cross-check based on per-
prompt Maximum Mean Discrepancy closely agrees with the sentence-level mean distance (Spearman $\rho=0.98$) and recovers the same qualitative findings. We organize these comparisons through a measure-theoretic lens making their
alignment and invariance assumptions explicit. We also establish an architecture-agnostic sufficient condition linking behavioral similarity to inference-
prompt coverage, small excess population log-loss, and similar effective target distributions—a possible training-side account rather than an empirical explanation of the observed trends. Our pipeline is label-free, and re-encoding every response with three further encoders—down to one $73\times$ smaller—preserves the rank geometry, the outliers, and the sign of the time trend.
Read the full abstract ▾
Computation and Language
Data Attribution of Emergent Misalignment with Persona Features
Clemens Vetter, David Kaczér, Lucie Flek et al. (+1 more)
Emergent misalignment (EM) is the phenomenon where fine-tuning a
language model on a narrow task leads to harmful behavior in unrelated domains. A leading mechanistic account attributes EM to persona features: latent directions acquired during pre-training that misaligned fine-tuning amplifies. We ask where these features come from: which pre-training documents activate them, and whether naturally occurring human-written text suffices to induce EM. Using Sparse Autoencoder (SAE) based model diffing across four open-weight models, we find that features related to jailbreak personas, sarcasm, deception, and manipulation are amplified by misalignment fine-tuning, while safety-relevant and assistant-identity features are suppressed. Steering individual features controls EM in both directions: it induces misalignment rates of up to 62% in aligned models – exceeding the 35% reached by misalignment fine-tuning itself – and re-aligns misaligned models to near-baseline misalignment rates. Attributing the causal features to a corpus of one million pre-training web documents retrieves semantically relevant narratives about villainous characters, domination, and harmful agency. However, fine-tuning on these human-written documents does not reliably induce EM, even after reformatting into assistant-style responses, whereas synthetic instruction-response pairs derived from the same content do – and transfer across model families. Semantic relevance alone is therefore not sufficient: response structure or model-generated phrasing plays an important role in inducing EM.
Read the full abstract ▾
All authors Clemens Vetter, David Kaczér, Lucie Flek, Florian Mai
Computation and Language
Templated or fully synthetic? Prompt construction as a confound in measuring LLM political stance beyond writing assistance
Ilias Chalkidis
Political stance detection in LLMs has long been dominated by closed-ended, multiple-choice political survey questions—originally designed for humans, and thus lacks the realism and nuance of human-AI interactions in the wild, while also being susceptible to sandbagging. The recent IssueBench framework substantially mitigates these limitations with templated
prompts anchored in real-world chat logs. Given the rise in non-work-related use of GenAI assistants, we extend IssueBench beyond writing assistance to include two additional tasks, information seeking and opinion sharing. We argue that templated
prompts still lack the nuance of real ones, especially for open-ended tasks, and remain recognisable as evaluation artefacts. We propose the use of fully synthetic (LLM-generated)
prompts, produced under detailed instructions with real
prompts as seeds. We assess the ecological validity of real, templated, and LLM-generated
prompts in a small-scale study covering 3 highly contested policy issues and 3 recent geopolitical conflicts. Human and LLM annotators rank LLM-generated
prompts as no less realistic than real ones and clearly more realistic than templated ones, and find that they carry their intended intent and stance more clearly; the LLMs separate templated
prompts from the other two far more sharply than the humans do. In a case study, templated and LLM-generated
prompts yield systematically different stance estimates for the same model, most visibly under neutral framings, where templated
prompts overstate the model's leaning in the direction encoded by the topic-and-stance text (filler) slotted into their templates.
Read the full abstract ▾
Computation and Language
On the Limitations of Cross-Lingual Consistency in Multilingual Text-to-image Generation
Sicheng Zhang, Zhonghao Yan, Binzhu Xie et al. (+4 more)
Text-to-image (T2I) generation has achieved remarkable progress in recent years. However, existing research has largely focused on English-only settings, leaving cross-lingual performance gaps and
language-specific effects insufficiently explored. To fill this gap, we introduce LingT2I, a
benchmark covering 10 widely used
languages with 33K
prompts, designed to evaluate cross-lingual effects in both content generation and text rendering. Building on this
benchmark, we conduct a comprehensive cross-lingual analysis, uncovering linguistic inequality and
language-dependent trade-offs across evaluation dimensions. Beyond quantitative evaluation, we further reveal a range of
language-dependent generation patterns, highlighting how linguistic factors and their corresponding cultural contexts systematically impact model outputs. Our
benchmark and analysis provide a foundation for studying cross-lingual behavior in T2I generation and facilitate the development of more robust and inclusive models. Code and
dataset are available at https://github.com/RISys-Lab/LingT2I.
Read the full abstract ▾
All authors Sicheng Zhang, Zhonghao Yan, Binzhu Xie, Shi Qiu, Muzammal Naseer, Naveed Akhtar, Mubarak Shah
Computation and Language
ConRub-Med: Reinforcement Learning with Consensus Rubrics for Open-Ended Medical Question Answering
Taojie Zhu, Yuan Xia, Tao Sun et al. (+8 more)
Reinforcement learning with verifiable rewards has been especially effective in mathematics and coding, where answers can be checked automatically. Many open-ended medical questions lack comparably cheap outcome verifiers: responses may be partly correct, incomplete, or contain clinically consequential errors. Rubrics written or validated by physicians offer strong clinical grounding, but involving experts in every instance is costly. Model-generated rubrics make this supervision scalable. We introduce ConRub-Med to preserve useful distinctions as rubric feedback moves from construction to policy optimization. For each
prompt, three heterogeneous
language models propose atomic criteria independently; a separate model reviews them, retaining only criteria with semantic support from all three generators. Three-State scoring distinguishes correct coverage, missing information, and incorrect claims. Errors receive negative rather than zero credit. When every response in a complete Group Relative Policy Optimization (GRPO) group receives the same final reward, a pairwise judge provides sequence advantages only if both candidate orders agree, without changing the scalar rewards. Groups without ties use vanilla GRPO. In a blinded study matched by question, two medical experts rate panels from the full pipeline as more clinically relevant than panels produced by one generator. Across the evaluated open models, ConRub-Med ranks first on six of nine
benchmarks and achieves the highest medical and generalization averages. Using the resulting rubric
dataset of 5,166
prompts, it scores $38.98 \pm 1.04$ (mean $\pm$ SD) on HealthBench-Hard, compared with InfiMed-ORBIT's 33.60 with 8,000 samples and 37.30 with 28,000.
Read the full abstract ▾
All authors Taojie Zhu, Yuan Xia, Tao Sun, Yizhi Wang, Yan Chen, Qunshan He, Tian Guan, Jian Wang, Jinjie Gu, Junwei Liu, Yonghong He
Computation and Language
What Iterated Self-Feeding Probes of Language Models Measure, and a test that separates the construction from the model
Nicolás Vera Zúñiga
A growing class of methods probes a
language model by feeding it its own output: self-consistency, iterated refinement, agentic loops. We ask what such a probe measures, in a construction chosen to make the question sharp: a ring of
token cells resampled in place by the model's own windowed conditional p_r(x_i | x_{i+-r}). The substrate is Glauber dynamics on
token sequences and is not new; what we change is the coupling. Advancing two rings that differ in one
token under common random numbers makes undamaged copies diverge by exactly zero, so damage spreading becomes measurable where a maximal coupling gives mixing times instead. The answer is that it measures two different things at once, in readings that look alike. Some quantities are fixed by the construction: the damage light cone is kinematic, and the radius scaling of the
token-space Lyapunov exponent lambda_ca(r) is model-invariant across 19 models and two scale ladders spanning 70x. Others genuinely track the model: lambda_ca crosses zero at a reproducible point in training, and the attractor share ranks models consistently however the
lattice is built. Left undistinguished, the first kind is readily mistaken for the second – we did so ourselves for four months, and report a phase transition we measured to three decimal places that belongs to the probe rather than to any
language model. We give the test that separates them: hold the construction fixed and vary the model, or hold the model fixed and vary the construction, and see which readings move. We validate the instrument by reproduction first, recovering a Domany-Kinzel damage field bit-exactly against an independent prediction, and we report the
estimator failures that this discipline caught – four retracted verdicts, each on a quantity that looked like a measurement. The methodology ships as a package.
Read the full abstract ▾
Computation and Language
MUSE: A Full-Text Cross-Domain Knowledge Base of Scientific Problems, Solutions, and Rationales
Tsofia Cohen and Tom Hope
Scientific papers contain fine-grained records of problem solving: authors mention technical obstacles and methods that were used to address them, often along with
reasoning on why those methods were chosen. We introduce MUSE (Mining Underlying Scientific Explanations), a full-text, multi-domain resource of scientific Problem-Solution-Rationale (P-S-R) triplets. We curate 579 expert-annotated full-text paragraphs, with a rich annotation schema covering salient problem, solution, and rationale spans, solves and rationale_of links and conceptual coreference. A modular extraction pipeline scales this annotation to build a high-quality knowledge base of 37K source-grounded P-S-R triplets. We evaluate the extraction components and include a preliminary experiment training a rationale-supervised LLM for scientific problem solving. Interestingly, we find that rationale supervision improves performance on complex, multi-constraint problems but can harm performance on simpler ones.
Read the full abstract ▾
Computation and Language
ReLTEx: Reliable LLM-based Taxonomy Expansion
Zeinab Ghamlouch and Mehwish Alam
Recent advances in Large
Language Models (LLMs) have demonstrated strong capabilities in generating semantically relevant concepts and relations, making them promising tools for taxonomy enrichment. However, directly relying on LLM-generated expansions often leads to noisy, redundant, or hierarchically inconsistent structures, limiting their reliability for automated taxonomy expansion. In this paper, we present ReLTEx, a framework for reliable LLM-based taxonomy expansion. ReLTEx combines LLM-driven candidate generation with structure-aware validation and recursive expansion control to improve the consistency and quality of generated taxonomies by reducing
hallucinations. We evaluate the proposed framework using
benchmark taxonomies under a masked taxonomy expansion setting and compare multiple validation strategies. Experimental results, supported by both adapted evaluation metrics and human evaluation, demonstrate that ReLTEx produces more reliable and semantically coherent taxonomy expansions.
Read the full abstract ▾
Computation and Language
REAP: Relation-Aware Elicitation and Parsing for Closed-Book Knowledge Base Construction from LLMs
Thanh-Dan Bui, Thanh-Trung Do and Tuan-Phong Nguyen
We present the REAP system for the AKBC Shared Task 2026 on constructing knowledge bases from
language models in a closed-book setting, subject to a budget of at most 32B parameters and no model fine-tuning. Our system combines structured chain-of-thought
reasoning, relation-specific query strategies, and a
reasoning-based empty-set gate to elicit parametric knowledge, followed by direct extraction into valid JSON arrays. On the test set, the system, built on the Mistral-Small-24B-Instruct-2501 model, achieves a macro-F1 score of 0.62, with particularly strong results on countryLandBordersCountry (F1 = 0.95), companyTradesAtStockExchange (F1 = 0.73), and hasArea (F1 = 0.77). Our code is publicly available at https://github.com/yammdd/AKBC-Shared-Task-2026.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
StreamFlow: Dynamic Memory Flows for Streaming Video Understanding
Muxin Fu, Yifan Zhang, Wentao Zhang et al. (+5 more)
Streaming
video understanding requires
multimodal large
language models (MLLMs) to preserve relevant evidence from continuously evolving streams under strict causality and bounded
memory. Yet existing paradigms remain limited: model-based methods require intrusive backbone updates, while
memory-based methods expend substantial visual-encoding computation on temporally redundant content and rely on rigid access to visual history. To address these limitations, we introduce StreamFlow, an efficient visual
memory framework that enables dynamic, on-demand access to historical visual information. StreamFlow combines a lightweight, dynamics-aware mid-term
memory that filters temporal redundancy before visual encoding with a latent long-term
memory that consolidates historical
video content into visual latents accessible to subsequent
reasoning. During generation, an
attention-guided
retrieval mechanism injects relevant visual latents when the model's reliance on visual evidence weakens. StreamFlow achieves state-of-the-art streaming
video understanding performance, reaching 67.73% overall accuracy on StreamingBench, while also delivering strong performance on offline long-
video benchmarks. Relative to the vanilla setting, it improves the visual
attention score (VAS) by 59.1% while reducing end-to-end latency and peak
memory by 50.4% and 21.1%, respectively, enabling more visually grounded and efficient
reasoning.
Read the full abstract ▾
All authors Muxin Fu, Yifan Zhang, Wentao Zhang, Fangming Guo, Qian Chen, Guibin Zhang, Shuicheng Yan, Bo An
Computation and Language
A Cost-Efficient Routing Pipeline for Multilingual Short-Text Classification Using Small Language Models
Wajdi Ben Saad and Safa Madiouni
Multilingual short-text classification supports operational systems such as content moderation, customer support routing, and intent recognition, yet aggregate evaluation often hides large differences between high-resource and low-resource
languages. Uniform inference policies are simple to deploy, but they assume that all
languages are equally well served. In this work, we evaluate a fixed-list routing strategy that keeps stronger
languages on a direct
multilingual path and selectively sends weaker
languages through translation into English before zero-shot classification. The pipeline is fully self-hosted, uses pretrained compact sentence encoders, and requires no task-specific fine-tuning. We test the approach on two
benchmarks chosen to differ in scale and label granularity: a 15-
language subset of SIB-200 for seven-way topic classification and a 15-locale subset of MASSIVE for intent classification over an official 60-intent inventory. On SIB-200, the best overall configuration is R1, which translates only the low-resource tier: high-tier and mid-tier Macro-F1 remain unchanged, while low-tier Macro-F1 rises from 0.4632 to 0.6828. On the MASSIVE subset, the same low-tier intervention raises low-tier Macro-F1 from 0.2143 to 0.4417, but the best overall result is obtained by full translation, R3, at Macro-F1 0.4647. Across these two
benchmarks, selective translation is a reliable intervention for weaker
languages, whereas the optimal routing boundary depends on the task. We therefore report routing through tier-level quality gains and tier-level latency rather than a single global efficiency score.
Read the full abstract ▾
Computation and Language
FaithformBench: Benchmarking Faithfulness of Mathematical Chain-of-Thought Autoformalisation
Rob Cornish, Iacopo Ghinassi, Po-Hung Yeh et al. (+7 more)
Autoformalisation (AF) systems map natural
language reasoning steps into formal statements in a proof assistant such as Lean. We consider how to assess the faithfulness of these systems. Existing approaches require expensive human-annotated ground truth, or rely on LLM judges or
embedding models, which come with limited guarantees of accuracy. In addition, these methods typically only consider inputs that are known to be correct, and therefore do not assess whether the AF translates incorrect inputs faithfully. To address these limitations, we propose a new
benchmark for AF faithfulness that is cheap to apply, sound under weak assumptions, and assesses both positive and negative examples. Our method is based on automatically generating perturbed
reasoning steps that are designed to be invalid, and then measuring validity preservation on unperturbed steps and invalidity preservation on perturbed steps. We apply our method to eight AF systems across four mathematical
datasets, and observe pervasive sycophancy: many AFs "silently correct" invalid inputs into provable statements. The most validity-preserving fine-tuned AFs are also the most sycophantic, suggesting a tension between validity and invalidity preservation in current AF systems.
Read the full abstract ▾
All authors Rob Cornish, Iacopo Ghinassi, Po-Hung Yeh, Shuqi Liu, Qiyuan Xu, Haoxuan Yin, Dominik Wagner, Wenda Li, Yee Whye Teh, Luke Ong
Computation and Language · Cross-listed from cs.CV
Order Matters: LVLMs as Judges for Temporal Reasoning in Image Sequences
Martina Ianaro, Guilherme Fernandes, Maurizio Gabbrielli et al. (+1 more)
As generative multimedia evolves from static image synthesis to complex, interleaved visual narratives, a foundational bottleneck has emerged: the judgment crisis. While human perception naturally synthesizes the temporal and logical flow of a story, automated evaluation systems remain largely "blind" to sequential continuity, often failing to distinguish between a coherent narrative and a semantically shuffled or contradictory sequence. This work identifies a critical structural gap in current
multimodal evaluation paradigms, arguing that the reliance on Large
Vision-
Language Models (LVLMs) as judges is fundamentally limited by architectural biases. Our analysis reveals a profound performance dichotomy: while models may appear competent in isolated pointwise scoring, they suffer a catastrophic collapse when required to perform pairwise discrimination of temporal order. We demonstrate that this is not merely a data-scarcity issue but a structural one. Through a series of diagnostic probes, we uncover systematic positional asymmetries, specifically primacy and recency effects, where a model's judgment of a story is significantly influenced by the placement of a frame, often more than by its semantic consistency. These biases, potentially rooted in causal masking and rotary
embeddings, suggest that current
transformer-based judges are inherently ill-equipped for long-form visual
reasoning. By exposing these blind spots, we challenge the multimedia community to move beyond snapshot-centric metrics and instead pioneer Temporally-Aware Evaluation paradigms that treat visual sequences as unified logical structures rather than unordered collections of frames.
Read the full abstract ▾
All authors Martina Ianaro, Guilherme Fernandes, Maurizio Gabbrielli, Joao Magalhaes
Computation and Language
Certify or Refuse: A Cross-Model Map for Selective Risk Control with Coverage Floors under Covariate Shift
Jiamiao Liu, Dewen Qiao, Yu Zhang et al. (+1 more)
Certified selective predictors attain whatever coverage they attain; operators impose an automation floor: answer at least a $\beta$-fraction of shifted target traffic with at most an $\alpha$-fraction of answers wrong. Under bounded-ratio
covariate shift we prove the Floor Certification Map: once that floor must be certified alongside the selection-conditioned risk $\alpha$, certification acquires a feasibility frontier and a two-resource complexity map, additive up to constants: risk in labeled source, the floor in unlabeled target samples. The rates are local, needing a regular frontier margin, slack below the local-regime threshold, and
lattice conditions: pre-registered with a
lattice margin for the upper bounds, compatible per-slack for the lower. The displayed split is the operational route; oracle weights also allow a labeled-source floor estimate. Three model-tagged results: a lower bound (Model-B), a matching oracle-weight upper bound (Model-A), and an implementable upper bound (Model-B') valid under a pre-registered exact stratified-shift model with nuisance cost priced explicitly. The match is across these models rather than a single-model minimax
theorem, and necessarily so: over the full bounded-ratio class no unknown-weight procedure matches at any sample size (Model-B is inconsistent, witnessed at $\alpha=\beta=1/2$). The nuisance's necessity is only partially settled. Complexity tracks a localized accepted-region functional, not global effective sample size (ESS), on both sides, though a fixed-ESS separation
theorem is left open; both lower-bound axes vanish as $\beta\to0$, so the floor creates the map. Empirically, the registered bite family diverges with log-log slope $-2.002$ within its pre-registered band; a 1,024-
cell audit records 0 violations where the formal certificates fire; and a single-corpus SQuAD-to-NewsQA feasibility audit returns honest refusal.
Read the full abstract ▾
All authors Jiamiao Liu, Dewen Qiao, Yu Zhang, Xuetao Chen