Computation and Language
Decoupling Generation and Selection for Budget-Constrained Faithful Summarization
Zeyu Wang, Guanghua Wang and Meng Xu
Abstractive summarization models remain vulnerable to factual inconsistency, redundancy, and weak length control. We propose a modular generation-and-selection framework for sentence-budget-constrained summarization. A pretrained generator produces multiple candidate summaries, which are decomposed into sentence-level candidates. A combinatorial selector then constructs the final summary by balancing relevance, factuality, and redundancy under an explicit budget. The framework supports MMR, ILP, and a DPP-inspired log-determinant objective without retraining the generator. Experiments on CNN/DailyMail, Multi-News, FaithBench, and TofuEval show consistent improvements in factuality and source-grounding metrics, especially for multi-document summarization, at the cost of lower reference-overlap scores. Human evaluation further indicates higher perceived consistency, relevance, clarity, and conciseness, with a small reduction in coherence. These results show that decoupling generation from selection provides a model-agnostic mechanism for improving factual grounding. Code is available at https://anonymous.4open.science/r/bcfs-D05E/.
Read the full abstract ▾
Computation and Language
LoopMTP: A looped transformer guided by latent multi-token prediction
Behzad Shomali, Markus Frey, David Berghaus et al. (+2 more)
Looped
transformers have emerged as a parameter-efficient alternative to scaling depth for strong
reasoning. By reusing one stack of layers across $T$ iterations, they attain the effective depth and
reasoning capabilities of larger models at a fixed parameter count. Yet existing approaches suffer from latent overthinking and undifferentiated computation, largely because intermediate representations receive no guidance across loops. Multi-
token prediction (MTP) supplies exactly the dense, forward-looking supervision the loop is missing. We propose LoopMTP, which links the two through a structural correspondence in latent space: a model that loops $T$ times can anticipate $T$ future
tokens. LoopMTP realizes this by softly aligning the hidden state of loop $t$ with the
embedding of the
token $t$ steps ahead, while a lightweight gate preserves useful information across iterations. LoopMTP improves average accuracy by up to 8.1% (relative) over the non-looped baseline, with training remaining stable for up to 15 loops.
Read the full abstract ▾
All authors Behzad Shomali, Markus Frey, David Berghaus, Joachim Koehler, Mehdi Ali
Computation and Language
A machine-readable catalogue of the Tsiolkovsky papers (fond 555, Archive of the Russian Academy of Sciences), and a way to measure how well its handwriting can be read
Vladimir Beskorovainyi
The personal archive of Konstantin Tsiolkovsky (1857-1935) is held as fond 555 of the Archive of the Russian Academy of Sciences. The archive scanned the fond and published the images, but with no queryable catalogue, no full-text search and no
dataset: the holdings can only be browsed one page at a time. This paper describes a machine-readable catalogue of all 2,019 files and 51,008 scans, a dating for 1,969 files taken from the archive's own descriptions, a page-level classification of every scan into handwriting and typescript, and a machine transcription of the fond in full: all 2,019 files and all 51,008 scans. It also reports a way to measure handwritten-text-recognition accuracy in an archive with no ground truth. Archives of the typewriter era often preserve one text twice, as manuscript and as a typed copy; transcribing both and comparing isolates the reading error, since source and pipeline are identical and only page difficulty differs. Across 1,759 such pairs from 224 files, two readings of a handwritten page agree on a median 37% of words and share a longest verbatim run of a median 10 words; the median is unchanged from the 294 pairs of the first version, on a sample six times larger. On two files that also have a published edition the estimate can be checked against ground truth: it is unbiased to within a percentage point and ranks pages as the truth does (rank correlation 0.92 where the edition is a faithful witness). This bounds use: two variants of one work here share 19% of words, below the rate at which two readings of a single page agree, so the redactions cannot be collated word by word at this quality. That negative result is reported as such, and the constraint is built into the tool.
Read the full abstract ▾
Computation and Language
Yuan Xie, Jiaqi Song, Xianliang Wang et al. (+3 more)
Modern LLM-based ASR systems have established
multilingual capability as a standard feature, leveraging large-scale
multilingual corpora and LLMs' cross-lingual knowledge to achieve competitive performance across
multilingual benchmarks. However, jointly modeling
languages with heterogeneous acoustic, phonological, and lexical characteristics inevitably introduces optimization conflicts, undermining
language-wise specialization. To address this challenge, we propose
Language-Specialized Multi-Teacher On-Policy
Distillation (LS-MOPD), which decouples
language-specific knowledge acquisition from
multilingual capability integration:
language-specialized teachers are independently optimized via reinforcement learning (RL), with their expertise then integrated into a generalist
multilingual student through
language routing and
token-level multi-teacher
distillation, thereby reducing direct cross-lingual optimization conflicts. We further explore static and dynamic acoustic-prefix configurations to examine how teacher-student prefix consistency influences the efficacy of on-policy
distillation. Experiments on
benchmarks covering Mandarin, Mandarin subdialects, Cantonese, and English demonstrate that LS-MOPD substantially outperforms RL baselines and surpasses the empirical performance envelope defined by the best-performing RL teachers on nearly all
benchmarks, revealing its potential to generalize beyond all teachers in
multilingual ASR.
Read the full abstract ▾
All authors Yuan Xie, Jiaqi Song, Xianliang Wang, Ming Lei, Jie Gao, Jie Wu
Computation and Language
Disentangling Language Modeling and Boundaries
Mykola Haltiuk
Byte-level
language models are usually argued for on the grounds of robustness,
multilingual fairness, and character-level skills. We point to a different, structural advantage: because they read and write bytes, any two of them share an output space, so knowledge transfer between them is exact and independent of how either was originally tokenized. We hypothesize that the two distributions a byte-level model produces, one over the next byte, one over where its patch boundaries fall, can be disentangled and changed almost independently. A model could absorb a teacher's capability while keeping its own boundaries, or change how it places those boundaries while keeping its capabilities. We lay out the two experiments that would settle the hypothesis, alongside preliminary measurements of the properties they rest on. We argue that the community should move toward a byte-level interface as a shared standard: if the hypothesis holds, then once byte-level models are the norm, transferring capabilities and reshaping boundaries between them become cheap and routine, free of the per-model tokenizer that blocks them today.
Read the full abstract ▾
Computation and Language
Looking under the Wrong Lamppost: On the Limitations of Automated Translation Quality Estimation
Serge Gladkoff, Angelika Vaasa, Sue Ellen Wright et al. (+2 more)
Automation of Translation Quality Estimation (QE) has emerged as a widely discussed approach to managing translation quality at scale, and a growing number of tools and technologies have been released in pursuit of this goal. However, the proliferation of new QE systems has not always been accompanied by robust, transparent, and reproducible research and testing. This gap deserves critical scrutiny. This paper examines some fundamental limitations of the QE technology from both theoretical and empirical perspectives, arguing that current QE systems are structurally ill-equipped to serve as reliable standalone tools in real-world translation workflows. The reviewed evidence suggests that QE suffers from a range of interrelated and largely unresolved limitations. Most fundamentally, the evaluation of the quality of translation at the level of isolated segments is problematic because it tends to miss out on cohesion, coherence, and stylistic and rhetorical text features. In addition, empirical research documents several other limitations and flaws, including failure to generalize, systematic biases, overfitting and distribution collapse, performance gaps, error annotation challenges, and data scarcity. These are structural limitations arising from the complexity of human
language and translation as a cognitive and communicative act - limitations that more data and better architectures have so far not overcome. Consequently, segment-level QE scores should not be used as a standalone basis for routing, release, or review bypass in production; we argue future work should focus on automating human evaluation grounded in MQM.
Read the full abstract ▾
All authors Serge Gladkoff, Angelika Vaasa, Sue Ellen Wright, Ingemar Strandvik, Lifeng Han
Computation and Language
SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs
Kejian Zhu, Zhuoran Jin, Shangqing Tu et al. (+5 more)
Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) exhibit fundamentally different behaviors in enhancing multi-task
reasoning for large
language models (LLMs). Our preliminary experiments revealed a phenomenon: SFT suffers from severe task conflicts under multi-stage training, whereas RL enables stable coexistence across diverse tasks. Empirically, we trace this to the parameter level, observing that RL induces sparse and approximately orthogonal updates across tasks. We provide a theoretical explanation for this mechanism by analyzing multi-task gradient interference. Our results reveal a distinction: interference in SFT is norm-limited, scaling with the absolute gradient magnitude, whereas interference in RL is variance-limited, bounded by the gradient variance induced by advantage normalization and on-policy optimization. This small variance bound yields near-orthogonal optimization directions across tasks. Leveraging this insight, we propose Parallel-RL, a paradigm that decouples multi-task training, significantly improving efficiency and flexibility.
Read the full abstract ▾
All authors Kejian Zhu, Zhuoran Jin, Shangqing Tu, Hongbang Yuan, Yushi Bai, Kang Liu, Juanzi Li, Jun Zhao
Computation and Language
Hi-TTRL: Regulating Consensus with Hints for Test-Time Reinforcement Learning
Kunbin Xu, Xingzuo Li, Xuefeng Bai et al. (+1 more)
Test-time reinforcement learning (TTRL) improves the
reasoning capabilities of large
language models without labeled data by updating the policy with pseudo-labels constructed through majority voting. While effective, the reward signal assigned from majority voting is highly sensitive to consensus strength, defined as the frequency of the most common answer within a rollout group. In TTRL, consensus strength plays a dual role: it reflects both the reliability of the pseudo-label and the distribution of advantages. Low consensus can amplify updates from unreliable pseudo-labels through disproportionately large advantages, whereas high consensus reduces reward contrast and ultimately yields vanishing gradients. In this paper, we introduce Hi-TTRL, a test-time reinforcement learning framework that utilizes hints during sampling to regulate rollout consensus strength. Hi-TTRL first estimates consensus strength from a partial rollout group. When the consensus strength falls outside a target interval, it invokes a Markov chain Monte Carlo (MCMC) hint sampler. The sampler targets the power-transformed prefix distribution and uses finite-step approximate sampling to generate rollout prefixes as hints. By tuning the power exponent, Hi-TTRL generates hints with a sharpened or flattened power target, steering rollout consensus strength toward the target interval. Experiments on multiple
datasets and backbones show that Hi-TTRL consistently improves over standard TTRL, with ablations and consensus-steering analyses validating the effectiveness of adaptive hint-guided consensus regulation.
Read the full abstract ▾
All authors Kunbin Xu, Xingzuo Li, Xuefeng Bai, Kehai Chen
Computation and Language
Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili
Ruolei Zhang, Teddy Njuguna and Yue Feng
Large
language models are increasingly deployed in
multilingual contexts, yet safety
alignment and bias evaluation remain overwhelmingly English-centric. We investigate whether social biases generalise across
languages by submitting 4,900 symmetric English–Swahili
prompt pairs to GPT-5.2 and Gemini 2.5 Flash across nine demographic bias axes, yielding 19,600 completions evaluated for stereotype prevalence, sentiment, refusal behaviour, and cross-lingual semantic similarity. Our findings show that bias transforms rather than transfers: stereotype rates shifted by up to 12 percentage points on specific axes, Gemini's neutral-sentiment rate doubled in Swahili, and GPT-5.2 refused 169
prompts in English and zero in Swahili, consistent with refusal behaviour anchored to English-
language surface forms at the behavioural level. Over 55% of
prompt pairs produced semantically dissimilar completions across both models. These reinforce the idea that English-only bias audits do not produce adequate coverage for
multilingual deployment.
Read the full abstract ▾
Computation and Language
Consensus Measures for Unstructured Biomedical Text Annotations
Pascal Wullschleger, Christian Kreis, Martin A. Walter et al. (+2 more)
Biomedical literature is increasingly mined for knowledge beyond the questions it was written to answer. Because the target concepts are not known in advance, annotators prefer open-ended labels, whose agreement is hard to quantify. We study soft inter-rater reliability for annotators providing unstructured texts for biomedical annotation tasks. Synthetic experiments show that soft reliability can be quantified using a variety of semantic equivalence measures, and that the choice of measure affects failure modes of the estimation.
Embeddings are scalable, but limited when differentiating similar but distinct concepts. Large
language models are promising, but limited by scalability for estimating agreement by chance. Finally, we suggest measures based on natural
language inference as a sensible compromise.
Read the full abstract ▾
All authors Pascal Wullschleger, Christian Kreis, Martin A. Walter, Jennifer Foster, Marc Pouly
Computation and Language · Cross-listed from cs.IR
Training Documents Reranker with Search Rubrics for Deep Research Agent
Wenhan Liu, Yu Lu, Qiaolin Xia et al. (+8 more)
Retrieval systems help deep research
agents generate high-quality answers by providing relevant documents. However, existing retrievers typically select documents through relevance matching, while individually well-matched top-$k$ documents may not form a set that satisfies the complex information needs of an
agent query (\eg, diverse, concise and authoritative documents). In this paper, we propose search-oriented rubrics that explicitly define the requirements that high-quality document sets should satisfy for each
agent query. Our search rubrics are organized into a hierarchical structure and synthesized using a powerful LLM. Based on these search rubrics, we further train a document reranker RubricRanker to select a high-quality subset from retrieved documents. We design a two-stage training framework that consists of rubrics-guided supervised fine-tuning and rubric-based reinforcement learning. Extensive experiments demonstrate that RubricRanker outperforms the strongest baseline by 2.6 points on four deep research
benchmarks and generalizes well to five RAG
benchmarks.
Read the full abstract ▾
All authors Wenhan Liu, Yu Lu, Qiaolin Xia, Hui Xu, Tong Zhao, Jian Xi, Yutao Zhu, Haijin Liang, Haibo Shi, Hao Wang, Zhicheng Dou
Computation and Language
ChronoLens: Measuring Language Change Across Time, Languages, and Linguistic Levels
Gagan Bhatia, Julian Schlenker, Simone Paolo Ponzetto et al. (+1 more)
Historical
language change affects morphology, syntax, semantics, and pragmatics, yet computational studies typically examine these levels with incompatible representations and therefore cannot determine whether they evolve together across
languages. We address this problem by asking how the magnitude and direction of change vary across linguistic levels,
languages, and historical periods within a single analytical space. We introduce ChronoLens, a framework that combines frozen
multilingual language models, feature-aligned crosscoders, and post-hoc linguistic interventions, and apply it to 44.98 million documents and approximately 17.2 billion
tokens from five parliamentary traditions spanning 1803–2026. The resulting sparse representations agree substantially more strongly with linguistic statistics than dense
embeddings or a pooled sparse autoencoder ($\rho=0.72$ versus $0.29$ and $0.28$), and reveal that morphology, syntax, semantics, and pragmatics generally change by comparable amounts within a
language, while
languages differ markedly in when, how far, and in which direction they change. These findings show that historical
language change is a structured, multidimensional process: similar magnitudes can conceal different trajectories, and meaningful cross-linguistic comparison requires measuring both distance and direction.
Read the full abstract ▾
All authors Gagan Bhatia, Julian Schlenker, Simone Paolo Ponzetto, Steffen Eger
Computation and Language
ConlangBench: Exploring Language Knowledge and Learning in LLMs through Diverse Constructed Languages
Jinhong Jeong, Seungyeop Yi, Sangah Lee et al. (+1 more)
Constructed
languages (conlangs) are intentionally created human
languages with a rich tradition of linguistic creativity. Despite their potential for studying
language learning in large
language models (LLMs), existing conlangs remain largely underexplored in LLM research. We present ConlangBench, the first large-scale
benchmark for evaluating and training LLMs on 21 existing conlangs. We collect over 21M conlang-English parallel sentence pairs (including 430K pairs across the 20 non-Esperanto conlangs) and 321K vocabulary entries. In bidirectional translation experiments, we find that models perform better on a posteriori conlangs, whose vocabularies are derived from natural
languages, reflecting the design characteristics of conlangs. Training on ConlangBench also shows that models can learn all eight conlangs for which sufficient parallel corpora are available, while their learning curves vary depending on how the conlangs were created. Our findings suggest that conlangs provide a unique testbed for investigating how LLMs acquire low-resource
languages.
Read the full abstract ▾
All authors Jinhong Jeong, Seungyeop Yi, Sangah Lee, Youngjae Yu
Computation and Language
Beyond Initialization Loss: A Systematic Study of Token Embedding Initialization Strategies for LLM Vocabulary Extension
Raviraj Joshi, Utkarsh Vaidya, Sanjay Singh Chauhan et al. (+1 more)
Vocabulary extension is an efficient way to adapt pretrained large
language models (LLMs) to new
languages, but the initialization of newly added
token embeddings can strongly affect continued pre-training (CPT) efficiency. We present a systematic study of more than 20 initialization strategies for Hindi vocabulary extension in Nemotron-3-Nano-30B-A3B. Our comparison spans vocabulary-averaging baselines; external and learned initialization methods, including FOCUS, top-k semantic
retrieval, and residual MLP mappings; subword composition; norm calibration; and input-output asymmetry. We find that subword composition methods outperform both vocabulary averaging and external/learned initialization approaches. Within subword composition, asymmetric variants achieve the lowest observed early validation loss and reveal distinct preferences for input and output
embedding initialization. The best observed configuration initializes the input
embedding matrix with uniform subword averaging and Hindi-specific norm calibration, and the output
language modeling head with character-length-weighted subword averaging. Relative to the standard Mean-all baseline, this full initialization pipeline reaches comparable validation loss with over a 6x reduction in CPT steps and exceeds the baseline's 3,500-step MILU-Hindi accuracy after only 500 steps. Finally, we show that initialization loss and initialization bits-per-byte (Init BPB) are unreliable predictors of downstream convergence, whereas lightweight CPT, as few as 50 steps, provides a cost-effective and reliable signal for selecting the best initialization strategy.
Read the full abstract ▾
All authors Raviraj Joshi, Utkarsh Vaidya, Sanjay Singh Chauhan, Niranjan Wartikar
Computation and Language
Efficient Multilingual Neural Machine Translation via Corpus-Driven Vocabulary Pruning: An English-Arabic Case Study
Ahmed Amine Aliane, Nasredine Semmar and Hassina Aliane
The adoption of large pre-trained
multilingual models for neural machine translation (MNMT) faces a major challenge: excessive
memory and computational consumption due to overly large vocabularies and
embedding layers. Although existing compression methods like
pruning, quantization and knowledge
distillation reduce parameter redundancy, they mainly preserve the structure of the original vocabulary, thereby leaving a major source of inefficiency unresolved. We propose in this paper a general optimization framework that combines a vocabulary
pruning method with a targeted fine-tuning protocol for MNMT models. We evaluate the proposed framework using three models (M2M100, NLLB-200, mBART-50) on the English-Arabic
language pair. Our approach reduces the vocabulary size from over 128,000 to approximately 10,000
tokens, enabling a 60%
memory saving without any loss in performance. Results show that optimized
multilingual models can match or exceed the performance of dedicated bilingual baselines. In particular, the pruned and fine-tuned M2M100 model achieves a competitive BLEU score of 42.04 (against 44.59 for the OPUS-MTen- ar bilingual model) while it significantly outperforms it on the COMET metric (0.8730 vs 0.7911) revealing superior semantic adequacy and fluency.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.MM
Adaptive Modality Reliability Diagnosis and Restoration for Robust Multimodal Intent Recognition
Suraj Kumar, Mohnish Raj, Soumi Chattopadhayay et al. (+2 more)
Multimodal intent recognition combines linguistic, acoustic, and visual evidence, but individual modalities may be noisy, missing, semantically conflicting, or disproportionately dominant. Existing methods typically infer modality importance implicitly and either reweight or suppress unreliable inputs, without determining whether a degraded modality can be repaired and subsequently trusted. We propose
PRIME (Precision-weighted Reliability Inference and Modality rEstoration), a closed-loop reliability guided framework that jointly diagnoses, restores, and reassesses modality quality at the sample level.
PRIME represents the weakness of each modality through a contextual log-variance estimated from complementary diagnostic evidence, including predictive confidence, epistemic disagreement, cross-modal consensus, and feature degeneracy. Because modality-reliability annotations are unavailable, the
estimator is explicitly trained using controlled modality corruption with known degradation severity, together with a heteroscedastic uncertainty objective. Rather than directly discarding an unreliable modality,
PRIME uses its estimated weakness to control a prototype-conditioned variational restoration module that reconstructs the degraded representation from complementary modalities. Crucially, reliability is re-estimated after restoration, allowing the model to determine whether the repaired representation has become sufficiently trustworthy to contribute to prediction. The resulting post-restoration precisions are used for inverse-variance
multimodal fusion. Experiments on
multimodal intent-recognition
benchmarks show that
PRIME maintains competitive clean-data performance while improving robustness under missing, noisy, conflicting, and modality-imbalanced conditions.
Read the full abstract ▾
All authors Suraj Kumar, Mohnish Raj, Soumi Chattopadhayay, Chandranath Adak, Ayan Dutta
Computation and Language · Cross-listed from cs.AI
ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation
Zhenghan Chen, Zekai Shao, Lidan Tan et al. (+10 more)
Annotations are essential to communicative visualization, helping explain data, emphasize key findings, and guide
attention. While
multimodal large
language models (MLLMs) offer new opportunities for automatic chart annotation authoring, their capabilities in this task remain underexplored. To address this gap, we introduce ChartAnno, a comprehensive
benchmark for evaluating MLLMs on chart annotation generation. ChartAnno contains 1,200 real-world charts with paired annotated and unannotated executable code, along with 3,600 annotation instructions spanning three levels of specificity. We also develop a multidimensional evaluation framework combining rule-based and LLM-judged metrics to assess execution, structural compliance, semantic consistency, and design effectiveness. We evaluate 10 representative MLLMs under two primary chart input settings: (1) chart code alone and (2) both code and chart image. Results reveal that proprietary models lead overall, though open-source models narrow the gap. While higher instruction specificity improves annotation quality, inferring abstract communicative intent remains difficult across all models. Providing chart images yields marginal benefit when code is available. We also examine the effect of chart code through an image-only ablation and analyze the effects of multiple task complexity indicators and instruction-level transitions. Further analyses characterize common failure modes and validate the reliability of the LLM-based judge. Experiments with D3 and SVG demonstrate the generalizability of ChartAnno beyond its primary Python setting.
Read the full abstract ▾
All authors Zhenghan Chen, Zekai Shao, Lidan Tan, Xin Lin, Xingchen Zeng, Yi Shan, Ziyue Lin, Xiaoliang Fu, Xinyuan Liu, Yuetong Guo, Fen Wang, Bongshin Lee, Siming Chen
Computation and Language
Probing Character-level Transformers for the Spanish L-shaped Morphome
Akhilesh Kakolu Ramarao, Kevin Tang, Wiebke Petersen et al. (+1 more)
When a
transformer learns an irregular morphological pattern, what has it learned? Our test case is the Spanish L-shaped morphome, a complex irregular pattern in which the verb's stem alternates in exactly the first-person singular indicative and all subjunctive forms, and whose membership no phonological, semantic, or syntactic feature predicts. Prior studies have shown that character-level
transformers can reproduce this pattern, but that evidence describes what models produce, not what they represent. Probing five architectures, twelve trained models each, under lemma-disjoint cross-validation with controls and surface baselines, we show that the models encode the L-shaped class itself, not just its visible alternations. It is decodable above every surface baseline, survives instances in which every form shows the same stem, and probes trained on alternating instances still classify non-alternating ones. The encoding is localized where the stem choice is made, at the stem-final consonant position of the middle decoder, before the alternant is read. And it is item-specific: which verbs a model learned matters far more than which architecture it is. The models store the morphome as an item-specific lexical abstraction, sufficient to reproduce the pattern but not to generalize it as humans do.
Read the full abstract ▾
All authors Akhilesh Kakolu Ramarao, Kevin Tang, Wiebke Petersen, Dinah Baer-Henney
Computation and Language · Cross-listed from cs.MM
Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs
Haoqian Kang, Liupeng Li, Kuofeng Gao et al. (+5 more)
Reasoning in
Multimodal Large
Language Models (MLLMs) requires both fine-grained visual perception and rigorous logical deduction. Explicit text-based Chain-of-Thought (CoT) is computationally expensive and prone to visual
hallucinations, while existing latent
reasoning methods typically require costly training. Furthermore, directly adapting training-free LLM
reasoning mechanisms to the
multimodal setting yields unstable performance. We identify that this failure stems from their reliance on
token-level
entropy, which fundamentally conflates perceptual ambiguity (e.g., unclear visual details) with logical uncertainty (e.g., complex
reasoning steps). To overcome this bottleneck, we present a novel training-free inference strategy for MLLMs that explicitly decouples perception and
reasoning. We propose a novel metric, the
vision-to-text
attention ratio, to dynamically gauge the model's cognitive focus. Guided by this metric, our proposed framework,
Attention-Guided Switching (AGS), adaptively triggers latent
reasoning for perceptual
tokens to preserve high-fidelity visual information in the continuous space, while enforcing explicit text generation for logical
tokens to maintain structural anchoring. Extensive experiments demonstrate that our method achieves state-of-the-art performance, significantly improving both accuracy and inference efficiency by reducing autoregressive steps and latency. Code is released at https://github.com/swordAndSnow/MM26-AGS.
Read the full abstract ▾
All authors Haoqian Kang, Liupeng Li, Kuofeng Gao, Jinpeng Wang, Zhenyu Lu, Bin Chen, Ke Chen, Yaowei Wang
Computation and Language
Predicting Multilingual Classification and Translation Performance of LLMs with Cross-Lingual Alignment – Is English Enough?
Adnan Al Ali, Kathy Hämmerl, Jindřich Libovický et al. (+1 more)
Multilingual large
language models (LLMs) have been shown to perform better on non-English classification tasks when the representations of the given
language are more aligned to English within the model. Several cross-lingual
alignment (CLA) scores have been proposed for use with LLMs, along with multiple approaches for extracting
embeddings from the models. We provide a comparative analysis of 27 CLA score variants, examining how they differ and how well each predicts downstream performance across three tasks. Crucially, while LLMs are widely used for generative tasks such as machine translation, prior work has focused almost exclusively on classification. We therefore investigate whether CLA scores are similarly predictive of translation performance. To enable computing correlations across target
languages, we propose a PMI-based translation metric, which is less dependent on the target
language and correlates strongly with chrF. We find that CLA with English predicts translation quality comparably to or better than source-target CLA, providing new evidence that LLMs use English as an internal pivot
language.
Read the full abstract ▾
All authors Adnan Al Ali, Kathy Hämmerl, Jindřich Libovický, Alexander Fraser
Computation and Language
Dynamically Allocating Evaluation Effort for Model Ranking
Vilém Zouhar, Julia Kreutzer, Alon Lavie et al. (+4 more)
While human evaluation is the gold standard in many NLP tasks, it suffers from prohibitive costs and poor scalability. When identifying top-performing models, typical evaluation protocols waste effort by exhaustively evaluating all models on the entire
benchmark, a safe but inefficient approach. In this work, we formalize multi-model human evaluation as a best-arm identification problem in a multi-armed bandit setup with correlated arms, where pulling an arm corresponds to human-evaluating a model. By sampling adaptively based on the intermediate model rankings obtained on the samples so far, we can focus the annotation budget on the most competitive models. We prove the optimality of the proposed
algorithms and show that it improves discrimination between top-performing models. This makes evaluations faster, cheaper and more aligned with large-scale competition evaluation goals.
Read the full abstract ▾
All authors Vilém Zouhar, Julia Kreutzer, Alon Lavie, Tom Kocmi, Matt Post, Ondřej Bojar, Mrinmaya Sachan
Computation and Language
DUD: Decoupled Update Dynamics for Reliable Uncertainty Quantification in Large Language Models
Yixin Bu, Runze Xia, Guanyun Zou et al. (+3 more)
Accurate Uncertainty Quantification (UQ) is critical for reliable deployment of Large
Language Models (LLMs), yet traditional probability-based metrics often fail to capture the model's true epistemic state. While recent mechanistic approaches leverage hidden state dynamics, they typically aggregate residual stream updates, conflating the distinct roles of parametric
memory (Feed-Forward Networks) and contextual processing (
Attention). We argue that this aggregation obscures fine-grained mechanistic conflicts, such as
memory-context misalignment, that are fundamental indicators of uncertainty. To address this, we introduce Decoupled Update Dynamics (DUD), a framework that explicitly decouples FFN and
Attention contributions via noise-induced causal interventions. By quantifying the independent restoration capabilities of each module, we construct a dual-stream dynamic profile that captures the model's internal fragility. Extensive experiments demonstrate that DUD significantly outperforms state-of-the-art baselines in both uncertainty estimation and calibration, while exhibiting superior cross-
dataset generalization, validating decoupled dynamics as a robust proxy for model faithfulness.
Read the full abstract ▾
All authors Yixin Bu, Runze Xia, Guanyun Zou, Yupeng Ji, Haodong Liu, Piji Li
Computation and Language
Don't Let Me Ask for It: LLMs Show Deficiencies in Active Multi-Turn Information Acquisition for Abductive Inference
Shahrukh Mohiuddin, Chalamalasetti Kranti, Sherzod Hakimov et al. (+1 more)
Abductive
reasoning requires forming hypotheses that explain observed evidence and revising them as new evidence becomes available. While large
language models (LLMs) are often evaluated on whether they solve abductive
reasoning tasks correctly, less is known about how they acquire evidence, update their hypotheses, and decide when to stop. We introduce Alien Abduction game, an interactive probe for studying these behaviours under different interaction modes. The modes vary in whether evidence is provided upfront or across turns, and whether queries are selected by the model or examples are provided by the oracle. Across models, providing evidence upfront leads to higher success rates than distributing it across turns. In multi-turn settings, some models commit before using the available evidence, while others exhaust the turn budget without converging. Models also achieve higher success rates when examples are provided by the oracle than when they select their own queries, although their final hypotheses are more consistent with the evidence they selected. These findings suggest that models may form hypotheses that fit self-selected evidence without sufficiently distinguishing them from alternatives, and may struggle to validate and refine their hypotheses or determine when to stop.
Read the full abstract ▾
All authors Shahrukh Mohiuddin, Chalamalasetti Kranti, Sherzod Hakimov, David Schlangen
Computation and Language
FACTWASH: Catching AI Rewrites That Wash Hearsay into Fact
Alex Kwon
AI systems rewrite information constantly: conversations become stored
memories, documents become answers. The rewrite can keep a claim while washing away what made it checkable, who said it, how sure they were, when it held. We call that failure factwashing, and release factwash, an open-source write-time gate that catches it deterministically, with named flags and evidence rather than an LLM judge. Building it answers a practical question: when does a cheap check suffice, and when do you need a model? What decides is whether the property has a bounded surface-cue inventory. Explicit negation cues are close to enumerable, so a word list finishes and transfers, reaching 0.91 F1 on untuned text. Hedging and attribution have open-ended realizations, so vocabulary plateaus near half recall, and a one-question LLM witness recovers +17 and +15 points of cue-detection recall at equal precision. Deployed, that witness may only lower a verdict, so it buys precision rather than coverage. We measure cue detection on 105,596 independently annotated sentences. A blind-labelled corpus of
memory writes then locates the failure: 55% of bad writes in conversational hearsay, 7% in business email (p < 0.001), so the first deployment question is not which detector to use but whether the failure occurs at all. On unmodified mem0 2.0.7, the gate flags 5 of 8 hedged-hearsay writes.
Read the full abstract ▾
Computation and Language
ArtECulture: Benchmarking Culture-Conditioned Visual Emotion Understanding in Multimodal Large Language Models
Xiaolin Chen, Xuemeng Song, Wenhao Shi et al. (+3 more)
Existing visual
emotion understanding methods typically ignore cultural variations in emotional perception. We introduce culture-conditioned visual
emotion understanding, a task that predicts the culture-specific emotional perception of a given image and explains the underlying rationale. Although related
benchmarks exist, they are limited by inconsistent individual annotations, which hinder the derivation of majority-supported culture-level
emotion labels, and imbalanced cultural coverage. Thus, we present ArtECulture, a
benchmark containing 6,792 artworks with culture-specific
emotion labels and explanations across English, Chinese, and Arabic cultures, with balanced Western and non-Western content. Evaluations of 16 open- and closed-source
Multimodal Large
Language Models (MLLMs) under a zero-shot setting reveal that the task remains challenging, with the best model achieving below 50% accuracy. To address this limitation, we introduce a
retrieval-augmented culture-conditioned
emotion understanding framework, which leverages a concept-based cultural
emotion knowledge base to inject explicit cultural knowledge into MLLMs without additional training. The framework improves both culturally aligned
emotion prediction and grounded explanation generation. Our
benchmark and code will be publicly released.
Read the full abstract ▾
All authors Xiaolin Chen, Xuemeng Song, Wenhao Shi, Xianjing Han, Mong-Li Lee, Wynne Hsu
Computation and Language
Benchmarking the Benchmarks: Testing the Predictive Validity of Commonsense Benchmarks
Ine Gevers and Walter Daelemans
Predicting LLM's capabilities on real-world tasks is essential, yet the extent to which performance on commonsense
benchmarks predicts downstream performance remains underspecified. To establish the practical usability of widely adopted commonsense
benchmarks, we evaluate 23 models from six families on four established commonsense
benchmarks, four reworked variants, three non-commonsense controls, and eight downstream tasks requiring implicit social, pragmatic, temporal, or physical
reasoning. We compare model rankings, compute controlled correlations, and use leave-one-family-out cross-validation to assess the criterion validity of commonsense
benchmarks. Our results show that revised
benchmarks largely preserve original model rankings and do not improve downstream predictive power. Commonsense
benchmarks show consistent cross-family predictive validity for only a narrow subset of downstream tasks, with smaller or metric-specific gains elsewhere. Overall, standardized commonsense
benchmarks provide task-dependent rather than broad evidence of downstream commonsense competence.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Distractor-Aware Truncation: Disentangling Context-Length Effects from Signal Loss in Long-Context LLM Benchmarks
Mohsen Arjmandi
A standard claim in the literature on
retrieval-augmented and
memory-augmented
language models is that shorter context is better when the relevant information is preserved. We test this claim by running every sample of two long-context
benchmarks – BABILong and GraphWalks (BFS) – at four context-retention fractions (100%, 75%, 50%, 25%) under two truncation protocols. The first is the naive protocol implicitly used in much prior work: drop content from the middle of the
prompt. The second is distractor-aware: identify the task-relevant content for each sample and drop only the rest. We evaluate three sizes of the Claude family (Haiku 4.5, Sonnet 4.6, Opus 4.7) and, to test cross-provider generality, GPT-5.5 from a different provider; we apply the same protocol to two further
benchmarks (MRCR v2, Oolong). Under naive truncation, score collapses monotonically (paired Wilcoxon, Holm-corrected p_adj < 0.05 in all eight BABILong and GraphWalks
cells). Under the distractor-aware protocol – which preserves the signal by construction – performance is preserved or improves: the two smaller Claude models show statistically significant gains on BABILong, while the larger models (Opus 4.7 and GPT-5.5) sit at their full-context ceiling. The naive collapse and its distractor-aware recovery replicate on GPT-5.5, ruling out a single-provider artifact. The mechanism is direct: under the naive protocol the answer-bearing content survives in fewer than 1% of samples at 25% retention; under the distractor-aware protocol it is preserved by construction. The naive protocol is therefore not a measurement of context-window effects; it is a measurement of how often middle-removal happens to spare the answer. We conclude that future studies of context-length effects must specify how they distinguish signal from distractor, or they are at best ambiguous between two opposite hypotheses.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics
Shashwat Sourav and Aishwarya Balwani
Chain-of-thought (CoT)
reasoning improves large
language model (LLM) performance while also providing an observable interface to the model's
reasoning process. Existing approaches that leverage verbalized CoTs to monitor
reasoning correctness, however, largely evaluate the semantic correctness or consistency of individual intermediate steps, rather than how the
reasoning process evolves across the trace. As a result, failures distributed across the
reasoning trajectory, rather than those localized to a single incorrect step, remain comparatively underexplored. Furthermore, verbalized CoTs need not faithfully reflect the model's internal
reasoning, motivating analyses that do not treat individual statements as literal accounts of internal computation. In this work, we therefore ask whether the dynamics of visible CoT can be leveraged to systematically distinguish successful from failed
reasoning without assuming such semantic faithfulness. We study a range of LLMs on verifiable Boolean satisfiability tasks with variable complexity, enabling controlled comparisons near each model's capability frontier. Tagging CoT sentences by
reasoning function reveals premature verification collapse on SAT problems: incorrect traces enter clause checking earlier, repeat similar operations, and finalize sooner. On UNSAT problems, models presumptuously move towards incorrect SAT conclusions, checking candidate assignments rather than deriving contradictions across constructed cases. Subsequently, a targeted proof-search
prompt intervention raises Llama3-70B accuracy from 13.3% to 85%, correcting 84.6% of these errors. These results show that capability failures can manifest as distributed, task-dependent changes in the structure of visible
reasoning, and that CoT dynamics agnostic to whether the verbalized trace reflects the model's internal computations can help diagnose and correct failures.
Read the full abstract ▾
Computation and Language
MoEGen: Mixture-of-Experts for Instance-Adaptive LoRA Generation
Yiming Zeng, Lei Lu, Zexin Li et al. (+10 more)
Parameter-efficient fine-tuning (PEFT) enables efficient adaptation of large
language models, but existing MoE-based PEFT methods typically improve capacity by storing multiple full LoRA experts, causing adapter storage to grow linearly with the number of experts and restricting adaptation to a fixed expert pool. We ask whether MoE-based PEFT can produce instance-specific adaptations without explicitly storing a separate LoRA module for each expert. To address this gap, we propose MoEGen, an adaptation framework that shifts MoE-based PEFT from expert selection to expert-conditioned parameter generation. Instead of storing each expert as a full LoRA adapter, MoEGen represents each expert as a small learnable vector, termed an expert code. It routes each input over these vectors and uses their weighted combination to condition a lightweight hypernetwork that generates input-specific low-rank updates. This design decouples expert capacity from adapter storage while enabling instance-conditioned adaptation. Experiments on eight commonsense
reasoning benchmarks show consistent improvements over strong static and MoE-based PEFT baselines across three backbones. MoEGen also performs strongly in joint medical and legal-domain adaptation.
Read the full abstract ▾
All authors Yiming Zeng, Lei Lu, Zexin Li, Zhuochun Li, Dehai Min, Shuoqiu Li, Shuyi Liao, Xidong Wu, Zeyu Zhang, Minmei Wang, Yu Zhao, Tingting Yu, Shangqian Gao
Computation and Language · Cross-listed from cs.CV
CIGTSurv: Clinical Information Guided Tri-modal Survival Prediction with Local Prototype Association and Global Feature Alignment
Jing Dai, Qibin Zhang, Weiwei Zhou et al. (+4 more)
Multimodal learning has significantly advanced survival prediction by integrating pathology images with genomic data. However, clinical information, despite its critical role in reflecting a patient' s overall health, remains underutilized due to its discrete, sparse, and low-dimensional nature. Furthermore, the inherent heterogeneity across these modalities pose significant challenges in modeling cross-modal interactions. In this paper, we propose CIGTSurv, a Clinical Information Guided Tri-modal framework for Survival prediction. Specifically, we first design a holistic text template and use pretrained foundation models to transform clinical tabular data into high-dimensional tokenized
embeddings. Using clinical information as an anchor, we then introduce a dual-level interaction mechanism: 1) a local prototype association (LPA) module based on cross-
attention to explicitly learn
token-level correspondences between different modalities, and 2) a global feature
alignment (GFA) loss based on Maximum Mean Discrepancy (MMD) to implicitly enhance cross-modal distribution consistency. Extensive experiments on five TCGA cancer cohorts demonstrate that CIGTSurv achieves state-of-the-art (SOTA) survival prediction performance. Our source code is publicly available at https://github.com/Daijing-ai/CIGT-Surv.git.
Read the full abstract ▾
All authors Jing Dai, Qibin Zhang, Weiwei Zhou, Mingde Xu, Jingsong Liu, Jingdong Zhang, Hongming Xu