Computation and Language · Cross-listed from cs.AI
Wenlong Wang and Fergal Reid
Recent work on recursive
reasoning models like TRM demonstrates that tiny networks (7M parameters) can achieve strong performance on abstract
reasoning tasks through latent recursion – iterative refinement in hidden representation space without emitting intermediate
tokens. This raises a natural question about operator choice: Mamba-2's state space recurrence is itself a form of iterative refinement, making it a natural candidate for recursive
reasoning – but does introducing Mamba-2 into the recursive scaffold preserve
reasoning capability? We investigate this by replacing the
Transformer blocks in TRM with Mamba-2 hybrid operators while maintaining parameter parity (6.83M vs 6.86M parameters). On ARC-AGI-1, we find that the hybrid improves pass@2 (the official metric) by +2.0% (45.88% vs 43.88%) and consistently outperforms at higher K values (+4.75% at pass@100), whilst maintaining pass@1 parity. This suggests improved candidate coverage – the model generates correct solutions more reliably – with similar top-1 selection. Our results validate that Mamba-2 hybrid operators preserve
reasoning capability within the recursive scaffold, establishing SSM-based operators as viable candidates in the recursive operator design space and taking a first step towards understanding the best mixing strategies for recursive
reasoning.
Read the full abstract ▾
Computation and Language
Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models
Xin Xu, Clive Bai, Kai Yang et al. (+8 more)
Large-scale verifiable
prompts underpin the success of Reinforcement Learning with Verifiable Rewards (RLVR), but they contain many uninformative examples and are costly to expand further. Recent studies focus on better exploiting limited training data by prioritizing hard
prompts whose rollout pass rate is 0. However, easy
prompts with a pass rate of 1 also become increasingly prevalent as training progresses, thereby reducing the effective data size. To mitigate this, we propose Composition-RL, an offline data-augmentation approach that composes existing
prompts before RL to address the pass-rate-1 bottleneck without modifying the RL optimizer. The composed
dataset is generated and verified once from the fixed source pool and then used for RL training. Extensive experiments across model sizes from 4B to 30B show that Composition-RL consistently improves
reasoning capability over RL trained on the original
dataset. Performance can be further boosted with a curriculum variant of Composition-RL that gradually increases compositional depth over training. Additionally, Composition-RL enables more effective cross-domain RL by composing
prompts drawn from different domains. All our code, data, and models are publicly available to facilitate future research.
Read the full abstract ▾
All authors Xin Xu, Clive Bai, Kai Yang, Tianhao Chen, Cai Yuchen, Yangkun Chen, Weijie Liu, Hao Chen, Yang Wang, Saiyong Yang, Can Yang
Computation and Language · Cross-listed from cs.CY
Artificial intelligence is creating a new global linguistic hierarchy
Giulia Occhini, Kumiko Tanaka-Ishii, Anna Barford et al. (+10 more)
Artificial intelligence (AI) has the potential to transform healthcare, education, governance and socioeconomic equity, but its benefits remain concentrated in a small number of
languages (Bender, 2019; Blasi et al., 2022; Joshi et al., 2020; Ranathunga and de Silva, 2022; Young, 2015).
Language AI - the technologies that underpin widely-used conversational systems such as ChatGPT - could provide major benefits if available in people's native
languages, yet most of the world's 7,000+ linguistic communities currently lack access and face persistent digital marginalization. Here we present a global longitudinal analysis of social, economic and infrastructural conditions across
languages to assess systemic inequalities in
language AI. We first analyze the existence of AI resources for 6003
languages. We find that despite efforts of the community to broaden the reach of
language technologies (Bapna et al., 2022; Costa-Jussà et al., 2022), the dominance of a handful of
languages is exacerbating disparities on an unprecedented scale, with divides widening exponentially rather than narrowing. Further, we contrast the longitudinal
diffusion of AI with that of earlier IT technologies, revealing a distinctive hype-driven pattern of spread. To translate our findings into practical insights and guide prioritization efforts, we introduce the
Language AI Readiness Index (EQUATE), which maps the state of technological, socio-economic, and infrastructural prerequisites for AI deployment across
languages. The index highlights communities where capacity exists but remains underutilized, and provides a framework for accelerating more equitable
diffusion of
language AI. Our work contributes to setting the baseline for a transition towards more sustainable and equitable
language technologies.
Read the full abstract ▾
All authors Giulia Occhini, Kumiko Tanaka-Ishii, Anna Barford, Refael Tikochinski, Songbo Hu, Roi Reichart, Yijie Zhou, ester James V. Miranda, Hannah Claus, Ulla Petti, Ivan Vulić, Ramit Debnath, Anna Korhonen
Computation and Language
Disentangling Ambiguity from Instability in Large Language Models: A Clinical Text-to-SQL Case Study
Angelo Ziletti and Leonardo D'Ambrosi
Deploying large
language models for clinical Text-to-SQL requires distinguishing two qualitatively different causes of output diversity: (i) input ambiguity that should trigger clarification, and (ii) model instability that should trigger human review. We propose CLUES, a framework that models Text-to-SQL as a two-stage process (interpretations –> answers) and decomposes semantic uncertainty into an ambiguity score and an instability score. The instability score is computed via the Schur complement of a bipartite semantic graph
matrix. Across AmbigQA/SituatedQA (gold interpretations) and a clinical Text-to-SQL
benchmark (known interpretations), CLUES improves failure prediction over state-of-the-art Kernel
Language Entropy. In deployment settings, it remains competitive while providing a diagnostic decomposition unavailable from a single score. The resulting uncertainty regimes map to targeted interventions - query refinement for ambiguity, model improvement for instability. The high-ambiguity/high-instability regime contains 51% of errors while covering 25% of queries, enabling efficient triage.
Read the full abstract ▾
Computation and Language
LaCy: What Small Language Models Can and Should Learn is Not Just a Question of Loss
Szilvia Ujváry, Louis Béthune, Pierre Ablin et al. (+3 more)
Language models have consistently grown to compress more world knowledge into their parameters, but the knowledge that can be pretrained into them is upper-bounded by their parameter size. Especially the capacity of Small
Language Models (SLMs) is limited, leading to factually incorrect generations. This problem is often mitigated by giving the SLM access to an outside source: the ability to query a larger model, documents, or a database. Under this setting, we study the fundamental question of which
tokens an SLM can and should learn during pretraining, versus which ones it should delegate via a <CALL>
token. We find that this is not simply a question of loss: although the loss is predictive of whether a predicted
token mismatches the ground-truth, it is insufficient for identifying which predictions would actually lead to factual or semantically invalid continuations. Some high-loss
tokens correspond to acceptable alternative continuations of a pretraining document and therefore should not trigger a <CALL>. This suggests that learnability cannot be characterized from loss alone, but requires additional domain-specific signals about the role of a
token in the sentence. In Wikipedia-like domains, we show that augmenting the loss signal with lightweight grammatical information from a spaCy parser substantially improves delegation decisions. Based on this insight, we propose LaCy, a novel pretraining method that combines loss with factuality signals to decide which
tokens an SLM should learn. Our experiments demonstrate that LaCy models successfully learn which
tokens to predict and when to call for help. This results in higher FactScores when generating in a cascade with a bigger model and outperforms Rho or LLM-judge trained SLMs, while being simpler and cheaper.
Read the full abstract ▾
All authors Szilvia Ujváry, Louis Béthune, Pierre Ablin, João Monteiro, Marco Cuturi, Michael Kirchhof
Computation and Language
Automatic Simplification of Common Vulnerabilities and Exposures Descriptions
Varpu Vehomäki and Kimmo K. Kaski
Understanding cyber security is increasingly important for individuals and organizations. However, a lot of information related to cyber security can be difficult to understand to those not familiar with the topic. In this study, we focus on investigating how large
language models (LLMs) could be utilized in automatic text simplification (ATS) of Common Vulnerability and Exposure (CVE) descriptions. Automatic text simplification has been studied in several contexts, such as medical, scientific, and news texts, but it has not yet been studied to simplify texts in the rapidly changing and complex domain of cyber security. We created a baseline for cyber security ATS and a test
dataset of 40 CVE descriptions, evaluated by two groups of cyber security experts in two survey rounds. We have found that while out-of-the box LLMs can make the text appear simpler, they struggle with meaning preservation. Code and data are available at https://version.aalto.fi/gitlab/vehomav1/simplification\_nmi.
Read the full abstract ▾
Computation and Language
DHPLT: large-scale multilingual diachronic corpora and word representations for semantic change modelling
Mariia Fedorova, Andrey Kutuzov and Khonzoda Umarova
In this resource paper, we present DHPLT, an open collection of diachronic corpora in 41 diverse
languages. DHPLT is based on the web-crawled HPLT
datasets; we use web crawl timestamps as the approximate signal of document creation time. The collection covers three time periods: 2011-2015, 2020-2021 and 2024-present (1 million documents per time period for each
language). We additionally provide pre-computed word type and
token embeddings and lexical substitutions for our chosen target words, while at the same time leaving it open for the other researchers to come up with their own target words using the same
datasets. DHPLT aims at filling in the current lack of
multilingual diachronic corpora for semantic change modelling (beyond a dozen of high-resource
languages). It opens the way for a variety of new experimental setups in this field. All the resources described in this paper are available at https://data.hplt-project.org/three/diachronic/, sorted by
language.
Read the full abstract ▾
Computation and Language
Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models
Yuzhe Shang, Pengzhi Gao, Wei Liu et al. (+2 more)
Open large
language models (LLMs) have demonstrated improving
multilingual capabilities in recent years. In this paper, we present a study of open LLMs for
multilingual machine translation (MT) across a range of
languages, and investigate the effects of model scaling and data scaling when adapting open LLMs to
multilingual MT through continual pretraining and instruction finetuning. Based on the Gemma3 model family, we develop MiLMMT-46, which achieves top-tier
multilingual translation performance across 46
languages. Extensive experiments show that MiLMMT-46 consistently outperforms recent state-of-the-art (SOTA) models, including Seed-X, HY-MT-1.5, and TranslateGemma, and achieves competitive performance with strong proprietary systems such as Google Translate and Gemini 3 Pro. Models are released at https://huggingface.co/collections/xiaomi-research/milmmt-46. Codes are released at https://github.com/xiaomi-research/gemmax.
Read the full abstract ▾
All authors Yuzhe Shang, Pengzhi Gao, Wei Liu, Jian Luan, Jinsong Su
Computation and Language · Cross-listed from cs.CV
Benchmarking Vision-Language Models for French PDF-to-Markdown Conversion
Bruno Rigal, Victor Dupriez, Alexis Mignon et al. (+2 more)
This report evaluates PDF-to-Markdown conversion using recent
Vision-
Language Models (VLMs) on challenging French documents. Document parsing is a critical step for
Retrieval-Augmented Generation (RAG) pipelines, where transcription and layout errors propagate to downstream
retrieval and grounding. Existing
benchmarks often emphasize English or Chinese and can over-penalize benign formatting and linearization choices (e.g., line breaks, list
segmentation, alternative table renderings) that are largely irrelevant for downstream use. We introduce a French-focused
benchmark of difficult pages selected via model-disagreement sampling from a corpus of 60{,}000 documents, covering handwritten forms, complex layouts, dense tables, and graphics-rich pages. Evaluation is performed with unit-test-style checks that target concrete failure modes (text presence, reading order, and local table constraints) combined with category-specific normalization designed to discount presentation-only variance. Across 15 models, we observe substantially higher robustness for the strongest proprietary models on handwriting and forms, while several open-weights systems remain competitive on standard printed layouts.
Read the full abstract ▾
All authors Bruno Rigal, Victor Dupriez, Alexis Mignon, Ronan Le Hy, Nicolas Mery
Computation and Language · Cross-listed from cs.LG
RAM-Net: Linear-Time Sequence Modeling with Sparsely Addressable State
Kaicheng Xiao, Haotian Li, Liran Dong et al. (+1 more)
Linear
attention offers an efficient alternative to full
attention with a fixed-size recurrent state. However, this state is shared by all
tokens, so information from distinct
tokens becomes superposed within it and produces inter-
token interference that degrades long-range fine-grained recall. To address this issue, we propose RAM-Net, which replaces dense access to a shared state with sparse address-based access. RAM-Net organizes the recurrent state as a fixed-size array of independent slots and uses an Address Decoder that maps each key or query into a sparse address, selecting a small subset of slots to write to or read from at each step. This design directs
tokens with non-overlapping addresses to disjoint slots, suppressing inter-
token interference, while keeping per-step state access dependent only on the number of selected slots rather than the total state size. Empirically, RAM-Net outperforms strong recurrent baselines on fine-grained long-range
retrieval and achieves the lowest perplexity with competitive commonsense
reasoning. It does so while accessing fewer state elements per step than all baselines, e.g., $8\times$ fewer than Mamba2.
Read the full abstract ▾
All authors Kaicheng Xiao, Haotian Li, Liran Dong, Guoliang Xing
Computation and Language
Do Large Language Models Adapt to Language Variation across Socioeconomic Status?
Elisa Bassignana, Mike Zhang, Dirk Hovy et al. (+1 more)
Humans adjust their linguistic style to the audience they are addressing. However, the extent to which LLMs adapt to different social contexts is largely unknown. As these models increasingly mediate human-to-human communication, their failure to adapt to diverse styles can perpetuate stereotypes and marginalize communities whose linguistic norms are less closely mirrored by the models, thereby reinforcing social stratification. We study the extent to which LLMs integrate into social media communication across different socioeconomic status (SES) communities. We collect a novel
dataset from Reddit and YouTube, stratified by SES. We
prompt four LLMs with incomplete text from that corpus and compare the LLM-generated completions to the originals along 94 sociolinguistic metrics, including syntactic, rhetorical, and lexical features. LLMs modulate their style with respect to SES to only a minor extent, often resulting in approximation or caricature, and tend to emulate the style of upper SES more effectively. Our findings (1) show how LLMs risk amplifying linguistic hierarchies and (2) call into question their validity for
agent-based social simulation, survey experiments, and any research relying on
language style as a social signal.
Read the full abstract ▾
All authors Elisa Bassignana, Mike Zhang, Dirk Hovy, Amanda Cercas Curry
Computation and Language
Who is the richest club in the championship? Detecting and Rewriting Underspecified Questions Improve QA Performance
Yunchong Huang, Gianni Barlacchi and Sandro Pezzelle
Large
language models (LLMs) perform well on well-posed questions, yet standard question-answering (QA)
benchmarks remain far from solved. We argue that this gap is partly due to underspecified questions - queries whose interpretation cannot be uniquely determined without additional context. To test this hypothesis, we introduce an LLM-based classifier to identify underspecified questions and apply it to several widely used QA
datasets, finding that 16% to over 50% of
benchmark questions are underspecified and that LLMs perform significantly worse on them. To isolate the effect of underspecification, we conduct a controlled rewriting experiment that serves as an upper-bound analysis, rewriting underspecified questions into fully specified variants while holding gold answers fixed. QA performance consistently improves under this setting, indicating that many apparent QA failures stem from question underspecification rather than model limitations. Our findings highlight underspecification as an important confound in QA evaluation and motivate greater
attention to question clarity in
benchmark design.
Read the full abstract ▾
Computation and Language
Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Text
Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes et al. (+1 more)
End-to-End
Speech Translation (E2E-ST) has seen significant advancements, yet current models are primarily benchmarked on curated, "clean"
datasets. This overlooks critical real-world challenges, such as morphological robustness to inflectional variations common in non-native or dialectal
speech. In this work, we adapt a text-based
adversarial attack targeting inflectional morphology to the
speech domain and demonstrate that state-of-the-art E2E-ST models are highly vulnerable it. While
adversarial training effectively mitigates such risks in text-based tasks, generating high-quality
adversarial speech data remains computationally expensive and technically challenging. To address this, we propose Cross-Modal Robustness Transfer (CMRT), a framework that transfers
adversarial robustness from the text modality to the
speech modality. Our method eliminates the requirement for
adversarial speech data during training. Extensive experiments across four
language pairs demonstrate that CMRT improves
adversarial robustness by an average of more than 3 BLEU points, establishing a new baseline for robust E2E-ST without the overhead of generating
adversarial speech.
Read the full abstract ▾
All authors Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
Computation and Language
AdaptEvolve: Improving Efficiency of Evolutionary AI Agents through Adaptive Model Selection
Pretam Ray, Pratik Prabhanjan Brahma, Zicheng Liu et al. (+1 more)
Evolutionary agentic systems intensify the trade-off between computational efficiency and
reasoning capability by repeatedly invoking large
language models (LLMs) during inference. This setting raises a central question: how can an
agent dynamically select an LLM that is sufficiently capable for the current generation step while remaining computationally efficient? While model cascades offer a practical mechanism for balancing this trade-off, existing routing strategies typically rely on static heuristics or external controllers and do not explicitly account for model uncertainty. We introduce AdaptEvolve: Adaptive LLM Selection for Multi-LLM Evolutionary Refinement within an evolutionary sequential refinement framework that leverages intrinsic generation confidence to estimate real-time solvability. Empirical results show that confidence-driven selection yields a favourable Pareto frontier, reducing total inference cost by an average of 37.9% across
benchmarks while retaining 97.5% of the upper-bound accuracy of static large-model baselines. Our code is available at https://github.com/raypretam/adaptive_llm_selection.
Read the full abstract ▾
All authors Pretam Ray, Pratik Prabhanjan Brahma, Zicheng Liu, Emad Barsoum
Computation and Language · Cross-listed from cs.AI
When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generation
Shani Goren, Ido Galil and Ran El-Yaniv
LLMs are widely used, yet they remain prone to factual errors that erode trust and limit adoption in high-risk settings. One approach to mitigate this risk is to equip models with uncertainty estimation mechanisms that abstain when confidence is low. However, this binary "all-or-nothing" approach is excessively restrictive in long-form settings, often discarding valuable information. We introduce Selective Abstraction (SA), a framework that enables LLMs to trade specificity for reliability by selectively reducing the detail of uncertain content. We formalize SA through selective risk and coverage, and propose Atom-wise Selective Abstraction, a claim-level instantiation that decomposes responses into atomic claims and replaces uncertain atoms with higher-confidence, less specific abstractions. We develop a novel end-to-end pipeline for evaluating long-form generation that instantiates risk as factual correctness and measures coverage as retained information. Our comprehensive empirical evaluation across five LLMs and two long-form factual
benchmarks shows that atom-wise SA consistently improves the risk-coverage trade-off, outperforming seven baselines, including claim removal, self-revision, sampling-based aggregation, and chain-of-verification. It improves AURC by up to 35.61% over claim removal, demonstrating that reducing specificity boosts reliability while preserving substantial information. We further provide a risk-guided procedure for selecting a confidence threshold that satisfies a target risk.
Read the full abstract ▾
Computation and Language
Benchmark Illusion: Disagreement among LLMs and Its Scientific Consequences
Eddie Yang and Dashun Wang
Benchmarks underpin how progress in large
language models (LLMs) is measured and trusted. Yet our analyses reveal that apparent convergence in
benchmark accuracy can conceal deep epistemic divergence. Using two major
reasoning benchmarks - MMLU-Pro and GPQA - we show that LLMs achieving comparable accuracy still disagree on 16-66% of items, and 16-38% among top-performing frontier models. These discrepancies suggest distinct error profiles for different LLMs. When such models are used for scientific data annotation and inference, their hidden disagreements propagate into research results: in re-analyses of published studies in education and political science, switching the annotation model can change estimated treatment effects by more than 80%, and in some cases reverses their sign. Together, these findings illustrate a
benchmark illusion, where equal accuracy may conceal disagreement, with model choice becoming a hidden yet consequential variable for scientific reproducibility.
Read the full abstract ▾
Computation and Language
LLM-based Triplet Extraction from Financial Reports
Dante Wesslund, Ville Stenström, Pontus Linde et al. (+1 more)
Corporate financial reports are a valuable source of structured knowledge for Knowledge Graph construction, but the lack of annotated ground truth in this domain makes evaluation difficult. We present a semi-automated pipeline for Subject-Predicate-Object triplet extraction that uses ontology-driven proxy metrics, specifically Ontology Conformance and Faithfulness, instead of ground-truth-based evaluation. We compare a static, manually engineered ontology against a fully automated, document-specific ontology induction approach across different LLMs and two corporate annual reports. The automatically induced ontology achieves 100% schema conformance in all configurations, eliminating the ontology drift observed with the manual approach. We also propose a hybrid verification strategy that combines regex matching with an LLM-as-a-judge check, reducing apparent subject
hallucination rates from 65.2% to 1.6% by filtering false positives caused by coreference resolution. Finally, we identify a systematic asymmetry between subject and object
hallucinations, which we attribute to passive constructions and omitted
agents in financial prose.
Read the full abstract ▾
All authors Dante Wesslund, Ville Stenström, Pontus Linde, Alexander Holmberg
Computation and Language
Towards Fair and Comprehensive Evaluation of Routers in Collaborative LLM Systems
Wanxing Wu, He Zhu, Yixia Li et al. (+7 more)
Large
language models (LLMs) have achieved success, but cost and
privacy constraints necessitate deploying smaller models locally while offloading complex queries to cloud-based models. Existing router evaluations are unsystematic, overlooking scenario-specific requirements and out-of-distribution robustness. We propose RouterXBench, a principled evaluation framework with three dimensions: router ability, scenario
alignment, and cross-domain robustness. Unlike prior work that relies on output probabilities or external
embeddings, we utilize internal hidden states that capture model uncertainty before answer generation. We introduce ProbeDirichlet, a lightweight router that aggregates cross-layer hidden states via learnable Dirichlet distributions with probabilistic training. Trained on multi-domain data, it generalizes robustly across in-domain and out-of-distribution scenarios. Our results show ProbeDirichlet achieves 16.68% and 18.86% relative improvements over the best baselines in router ability and high-accuracy scenarios, with consistent performance across model families, model scales, heterogeneous tasks, and agentic workflows.
Read the full abstract ▾
All authors Wanxing Wu, He Zhu, Yixia Li, Lei Yang, Jiehui Zhao, Hongru Wang, Jian Yang, Benyou Wang, Bingyi Jing, Guanhua Chen
Computation and Language
DMAP: A Distribution Map for Text
Tom Kempton, Julia Rozanova, Parameswaran Kamalaruban et al. (+5 more)
Large
Language Models (LLMs) are a powerful tool for statistical text analysis, with derived sequences of next-
token probability distributions offering a wealth of information. Extracting this signal typically relies on metrics such as perplexity, which do not adequately account for context; how one should interpret a given next-
token probability is dependent on the number of reasonable choices encoded by the shape of the conditional distribution. In this work, we present DMAP, a mathematically grounded method that maps a text, via a
language model, to a set of samples in the unit interval that jointly encode rank and probability information. This representation enables efficient, model-agnostic analysis and supports a range of applications. We illustrate its utility through three case studies: (i) validation of generation parameters to ensure data integrity, (ii) examining the role of probability
curvature in machine-generated text detection, and (iii) a forensic analysis revealing statistical fingerprints left in downstream models that have been subject to post-training on synthetic data. Our results demonstrate that DMAP offers a unified statistical view of text that is simple to compute on consumer hardware, widely applicable, and provides a foundation for further research into text analysis with LLMs.
Read the full abstract ▾
All authors Tom Kempton, Julia Rozanova, Parameswaran Kamalaruban, Maeve Madigan, Karolina Wresilo, Yoann L. Launay, David Sutton, Stuart Burrell
Computation and Language · Cross-listed from cs.LG
A$^{2}$V-SLP: Alignment-Aware Variational Modeling for Disentangled Sign Language Production
Sümeyye Meryem Taşyürek, Enis Mücahid İskender and Hacer Yalim Keles
Building upon recent structural disentanglement frameworks for sign
language production, we propose A$^{2}$V-SLP, an
alignment-aware variational framework that learns articulator-wise disentangled latent distributions rather than deterministic
embeddings. A disentangled Variational Autoencoder (VAE) encodes ground-truth sign pose sequences and extracts articulator-specific mean and variance vectors, which are used as distributional supervision for training a non-autoregressive
Transformer. Given text
embeddings, the
Transformer predicts both latent means and log-variances, while the VAE decoder reconstructs the final sign pose sequences through
stochastic sampling at the decoding stage. This formulation maintains articulator-level representations by avoiding deterministic latent collapse through distributional latent modeling. In addition, we integrate a gloss
attention mechanism to strengthen
alignment between linguistic input and articulated motion. Experimental results show consistent gains over deterministic latent regression, achieving state-of-the-art back-translation performance and improved motion realism in a fully gloss-free setting.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception
Lai Wei, Liangbo He, Jun Lan et al. (+9 more)
Multimodal Large
Language Models (MLLMs) excel at broad visual understanding but still struggle with fine-grained perception, where decisive evidence is small and easily overwhelmed by global context. Recent "Thinking-with-Images" methods alleviate this by iteratively zooming in and out regions of interest during inference, but incur high latency due to repeated tool calls and visual re-encoding. To address this, we propose Region-to-Image
Distillation, which transforms zooming from an inference-time tool into a training-time primitive, thereby internalizing the benefits of agentic zooming into a single forward pass of an MLLM. In particular, we first zoom in to micro-cropped regions to let strong teacher models generate high-quality VQA data, and then distill this region-grounded supervision back to the full image. After training on such data, the smaller student model improves "single-glance" fine-grained perception without tool use. To rigorously evaluate this capability, we further present ZoomBench, a hybrid-annotated
benchmark of 845 VQA data spanning six fine-grained perceptual dimensions, together with a dual-view protocol that quantifies the global–regional "zooming gap". Experiments show that our models achieve leading performance across multiple fine-grained perception
benchmarks, and also improve general
multimodal cognition on
benchmarks such as visual
reasoning and GUI
agents. We further discuss when "Thinking-with-Images" is necessary versus when its gains can be distilled into a single forward pass. Our code is available at https://github.com/inclusionAI/Zooming-without-Zooming.
Read the full abstract ▾
All authors Lai Wei, Liangbo He, Jun Lan, Lingzhong Dong, Yutong Cai, Siyuan Li, Huijia Zhu, Weiqiang Wang, Linghe Kong, Yue Wang, Zhuosheng Zhang, Weiran Huang
Computation and Language · Cross-listed from cs.AI
Prototype Transformer: Towards Language Model Architectures Interpretable by Design
Yordan Yordanov, Matteo Forasassi, Bayar Menzat et al. (+6 more)
While state-of-the-art
language models (LMs) surpass most humans in certain domains, their
reasoning remains largely opaque, reducing trust and increasing the risk of deception and
hallucination. We introduce the Prototype
Transformer (ProtoT), an autoregressive LM architecture that replaces the quadratic-cost self-
attention module of the
Transformer with a linear-cost module based on prototypes, which are learned parameter vectors. In ProtoT, prototypes create communication channels that aggregate contextual information at different time scales. We show that this structure leads prototypes to automatically capture nameable concepts, such as "woman", during training, offering a path toward interpreting model
reasoning and making targeted edits to model behavior. Compared with baselines, ProtoT scales well with model and data size, is robust to input perturbations, and performs well on text generation and downstream tasks, including GLUE. These results suggest that ProtoT is a promising step toward autoregressive
language models that are more interpretable by design.
Read the full abstract ▾
All authors Yordan Yordanov, Matteo Forasassi, Bayar Menzat, Ruizhi Wang, Chang Qi, Markus Kaltenberger, Amine M'Charrak, Tommaso Salvatori, Thomas Lukasiewicz
Computation and Language
A Subword Embedding Approach for Variation Detection in Luxembourgish User Comments
Anne-Marie Lutgen, Alistair Plum and Christoph Purschke
This paper presents an
embedding-based approach to detecting variation without relying on prior normalisation or predefined variant lists. The method trains subword
embeddings on raw text and groups related forms through combined cosine and n-gram similarity. This allows spelling and morphological diversity to be examined and analysed as linguistic structure rather than treated as noise. Using a large corpus of Luxembourgish user comments, the approach uncovers extensive lexical and orthographic variation that aligns with patterns described in dialectal and sociolinguistic research. The induced families capture systematic correspondences and highlight areas of regional and stylistic differentiation. The procedure does not strictly require manual annotation, but does produce transparent clusters that support both quantitative and qualitative analysis. The results demonstrate that distributional modelling can reveal meaningful patterns of variation even in ”noisy” or low-resource settings, offering a reproducible methodological framework for studying
language variety in
multilingual and small-
language contexts.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CR
More Haste, Less Speed: Weaker Single-Layer Watermark Improves Distortion-Free Watermark Ensembles
Ruibo Chen, Yihan Wu, Xuehao Cui et al. (+2 more)
Watermarking has emerged as a crucial technique for detecting and attributing content generated by large
language models. While recent advancements have utilized watermark ensembles to enhance robustness, prevailing methods typically prioritize maximizing the strength of the watermark at every individual layer. In this work, we identify a critical limitation in this "stronger-is-better" approach: strong watermarks significantly reduce the
entropy of the
token distribution, which paradoxically weakens the effectiveness of watermarking in subsequent layers. We theoretically and empirically show that detectability is bounded by
entropy and that watermark ensembles induce a monotonic decrease in both
entropy and the expected green-list ratio across layers. To address this inherent trade-off, we propose a general framework that utilizes weaker single-layer watermarks to preserve the
entropy required for effective multi-layer ensembling. Empirical evaluations demonstrate that this counter-intuitive strategy mitigates signal decay and consistently outperforms strong baselines in both detectability and robustness.
Read the full abstract ▾
All authors Ruibo Chen, Yihan Wu, Xuehao Cui, Jingqi Zhang, Heng Huang
Computation and Language · Cross-listed from cs.AI
Detecting RLVR Training Data via Structural Convergence of Reasoning
Hongbo Zhang, Yue Yang, Jianhao Yan et al. (+3 more)
Reinforcement learning with verifiable rewards (RLVR) is central to training modern
reasoning models, but the undisclosed training data raises concerns about
benchmark contamination. Unlike pretraining methods, which optimize models using
token-level probabilities, RLVR fine-tunes models based on reward feedback from self-generated
reasoning trajectories, making conventional likelihood-based detection methods less effective. We show that RLVR induces a distinctive behavioral signature:
prompts encountered during RLVR training result in more rigid and similar generations, while unseen
prompts retain greater diversity. We introduce Min-$k$NN Distance, a simple black-box detector that quantifies this collapse by sampling multiple completions for a given
prompt and computing the average of the $k$ smallest nearest-neighbor edit distances. Min-$k$NN Distance requires no access to the reference model or
token probabilities. Experiments across multiple RLVR-trained
reasoning models show that Min-$k$NN Distance reliably distinguishes RL-seen examples from unseen ones and outperforms existing membership inference and RL contamination detection baselines.
Read the full abstract ▾
All authors Hongbo Zhang, Yue Yang, Jianhao Yan, Guangsheng Bao, Yue Zhang, Yue Zhang
Computation and Language · Cross-listed from cs.AI
Beyond End-to-End Video Models: An LLM-Based Multi-Agent System for Educational Video Generation
Lingyong Yan, Jiulong Wu, Dong Xie et al. (+3 more)
Although recent end-to-end
video generation models demonstrate impressive performance in visually oriented content creation, they remain limited in scenarios that require strict logical rigor and precise knowledge representation, such as instructional and educational media. To address this problem, we propose LASEV, a hierarchical LLM-based multi-
agent system for generating high-quality instructional
videos from educational problems. LASEV formulates educational
video generation as a multi-objective task that simultaneously demands correct step-by-step
reasoning, pedagogically coherent narration, semantically faithful visual demonstrations, and precise audio–visual
alignment. To address the limitations of prior approaches–including low procedural fidelity, high production cost, and limited controllability–LASEV decomposes the generation workflow into specialized
agents that collaborate through a central Orchestrating
Agent, shared production state, explicit quality gates, and iterative critique mechanisms. Specifically, the Orchestrating
Agent supervises a Solution
Agent for rigorous problem solving, an Illustration
Agent that produces executable visualization code, and a Narration
Agent for learner-oriented instructional scripts. In addition, all outputs from the working
agents are subject to semantic critique, rule-based constraints, and tool-based compilation checks. Rather than directly synthesizing pixels, the system constructs a structured executable
video script that is deterministically compiled into synchronized visuals and narration using template-driven assembly rules, enabling fully automated production without manual editing. In large-scale deployments, LASEV achieves a throughput exceeding one million
videos per day, delivering over a 95% reduction in cost compared to current industry-standard approaches while maintaining a high acceptance rate.
Read the full abstract ▾
All authors Lingyong Yan, Jiulong Wu, Dong Xie, Weixian Shi, Deguo Xia, Jizhou Huang
Computation and Language · Cross-listed from cs.AI
TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agents
Aladin Djuhera, Swanand Kadhe, Farhan Ahmed et al. (+3 more)
Advances in large
language models (LLMs) are driving a shift toward using reinforcement learning (RL) to train
agents from iterative, multi-turn interactions across tasks. However, multi-turn RL remains challenging as rewards are often sparse or delayed, and environments can be
stochastic. In this regime, naive trajectory sampling can hinder exploitation and induce mode collapse. We propose TSR (Trajectory-Search Rollouts), a training-time approach that repurposes test-time scaling ideas for improved per-turn rollout generation. TSR performs lightweight tree-style search to construct higher-quality trajectories by selecting promising actions and trajectory prefixes during rollout generation. This improves rollout quality while preserving stable policy optimization and remains compatible with standard policy-gradient optimizers by design. Across Sokoban, FrozenLake, and WebShop, TSR achieves success-rate gains of up to 15 percentage points and converges in fewer optimization steps, while trading additional training-time rollout compute for stronger policies that require no search at inference time. By moving search from test time to the rollout stage of training, TSR provides a modular mechanism for stronger multi-turn
agent learning, complementary to existing frameworks and rejection-sampling-style selection methods.
Read the full abstract ▾
All authors Aladin Djuhera, Swanand Kadhe, Farhan Ahmed, Syed Zawad, Heiko Ludwig, Holger Boche
Computation and Language
MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
MiniCPM Team: Wenhao An, Yingfa Chen, Yewei Fang et al. (+43 more)
The
evolution of large
language models (LLMs) towards applications with ultra-long contexts faces challenges posed by the high computational and
memory costs of the
Transformer architecture. While existing sparse and linear
attention mechanisms attempt to mitigate these issues, they typically involve a trade-off between
memory efficiency and model performance. This paper introduces MiniCPM-SALA, a 9B-parameter hybrid architecture that integrates the high-fidelity long-context modeling of sparse
attention (InfLLM-V2) with the global efficiency of linear
attention (Lightning
Attention). By employing a layer selection
algorithm to integrate these mechanisms in a 1:3 ratio and utilizing a hybrid positional encoding (HyPE), the model maintains efficiency and performance for long-context tasks. Furthermore, we introduce a cost-effective continual training framework that transforms pre-trained
Transformer-based models into hybrid models, which reduces training costs by approximately 75% compared to training from scratch. Extensive experiments show that MiniCPM-SALA maintains general capabilities comparable to full-
attention models while offering improved efficiency. On a single NVIDIA A6000D GPU, the model achieves up to 3.5x the inference speed of the full-
attention model at the sequence length of 256K
tokens and supports context lengths of up to 1M
tokens, a scale where traditional full-
attention 8B models fail because of
memory constraints.
Read the full abstract ▾
All authors MiniCPM Team: Wenhao An, Yingfa Chen, Yewei Fang, Jiayi Li, Xin Li, Yaohui Li, Yishan Li, Yuxuan Li, Biyuan Lin, Chuan Liu, Hezi Liu, Siyuan Liu, Hongya Lyu, Yinxu Pan, Shixin Ren, Xingyu Shen, Zhou Su, Haojun Sun, Yangang Sun, Zhen Leng Thai, Xin Tian, Rui Wang, Xiaorong Wang, Yudong Wang, Bo Wu, Xiaoyue Xu, Dong Xu, Shuaikang Xue, Jiawei Yang, Bowen Zhang, Jinqian Zhang, Letian Zhang, Shengnan Zhang, Xinyu Zhang, Xinyuan Zhang, Zhu Zhang, Hengyu Zhao, Jiacheng Zhao, Zhi Zheng, Jie Zhou, Zihan Zhou, Shuo Wang, Chaojun Xiao, Xu Han, Zhiyuan Liu, Maosong Sun
Computation and Language
Think Longer to Explore Deeper: Learn to Explore In-Context via Length-Incentivized Reinforcement Learning
Futing Wang, Jianhao Yan, Yun Luo et al. (+6 more)
Achieving effective test-time scaling requires models to engage in In-Context Exploration – the intrinsic ability to generate, verify, and refine multiple
reasoning hypotheses within a single continuous context. Grounded in State Coverage theory, our analysis identifies a critical bottleneck to enabling this capability: while broader state coverage requires longer
reasoning trajectories, the probability of sampling such sequences decays exponentially during autoregressive generation, a phenomenon we term the “Shallow Exploration Trap”. To bridge this gap, we propose Length-Incentivized Exploration(\method). This simple yet effective recipe explicitly encourages models to explore more via a length-based reward coupled with a redundancy penalty, thereby maximizing state coverage in two-step manner. Comprehensive experiments across different models (Qwen3, Llama) demonstrate that \method effectively incentivize in-context exploration. As a result, our method achieves an average improvement of 4.4% on in-domain tasks and a 2.7% gain on out-of-domain
benchmarks.
Read the full abstract ▾
All authors Futing Wang, Jianhao Yan, Yun Luo, Ganqu Cui, Zhi Wang, Xiaoye Qu, Yue Zhang, Yu Cheng, Tao Lin
Computation and Language · Cross-listed from cs.CV
Mask What Matters: Mitigating Object Hallucinations in Multimodal Large Language Models with Object-Aligned Visual Contrastive Decoding
Boqi Chen, Xudong Liu and Jianing Qiu
We study object
hallucination in
Multimodal Large
Language Models (MLLMs) and improve visual contrastive decoding (VCD) by constructing an object-aligned auxiliary view. We leverage object-centric
attention in self-supervised
Vision Transformers. In particular, we remove the most salient visual evidence to construct an auxiliary view that disrupts unsupported
tokens and produces a stronger contrast signal. Our method is
prompt-agnostic, model-agnostic, and can be seamlessly plugged into the existing VCD pipeline with little computation overhead, i.e., a single cacheable forward pass. Empirically, our method demonstrates consistent gains on two popular object
hallucination benchmarks across two MLLMs.
Read the full abstract ▾