Computation and Language
Inducing Sustained Creativity and Diversity in Large Language Models
Queenie Luo, Gary King, Michael Puett et al. (+1 more)
We address a not-widely-recognized subset of exploratory search, where a user sets out on a typically long "search quest" for the perfect wedding dress, overlooked research topic, killer company idea, etc. The first few outputs of current large
language models (LLMs) may be helpful but only as a start, since the quest requires learning the search space and evaluating many diverse and creative alternatives along the way. Although LLMs encode an impressive fraction of the world's knowledge, common decoding methods are narrowly optimized for
prompts with correct answers and thus return mostly homogeneous and conventional results. Other approaches, including those designed to increase diversity across a small set of answers, start to repeat themselves long before search quest users learn enough to make final choices, or offer a uniform type of "creativity" to every user asking similar questions. We develop a novel, easy-to-implement decoding scheme that induces sustained creativity and diversity in LLMs, producing as many conceptually unique results as desired, even without access to the inner workings of an LLM's vector space. The
algorithm unlocks an LLM's vast knowledge, both orthodox and heterodox, well beyond modal decoding paths. With this approach, search quest users can more quickly explore the search space and find satisfying answers.
Read the full abstract ▾
All authors Queenie Luo, Gary King, Michael Puett, Michael D. Smith
Computation and Language
Beyond Scalar Rewards: Dense Feedback for LLM Policy Synthesis in Sequential Social Dilemmas
Víctor Gallego
We propose an LLM harness that generates code-based policy functions for multi-
agent environments, evaluates them with self-play, and refines them using feedback from previous iterations. Following the recent line of work in feedback engineering (the design of which information signals are shown to the LLM during refinement), we compare sparse feedback (scalar reward only) with dense feedback (reward plus social metrics: efficiency, equality, sustainability, peace). In two Sequential Social Dilemmas (Gathering and Cleanup) and with two frontier LLMs (Claude Sonnet 4.6, Gemini 3.1 Pro), dense feedback improves over or matches sparse feedback on all metrics. We explain this asymmetry via feedback aliasing: when the scalar reward maps distinct failure modes into the same value (e.g., under- vs. over-cleaning), social metrics disambiguate and allow the LLM to diagnose which direction of improvement to take. We conclude that social metrics act as a coordination signal, leading to strategies such as Voronoi territory partitioning and adaptive cleaner schedules. Code at https://github.com/vicgalle/llm-policies-social-dilemmas.
Read the full abstract ▾
Computation and Language
Vocabulary shapes cross-lingual variation of word-order learnability in language models
Jonas Mayer Martins, Jaap Jumelet, Viola Priesemann et al. (+1 more)
Why do some
languages like Czech permit free word order, while others like English do not? We address this question by pretraining
transformer language models on a spectrum of synthetic word-order variants of natural
languages. We observe that greater word-order irregularity consistently raises model surprisal, indicating reduced learnability. Sentence reversal, however, affects learnability only weakly. A coarse distinction of free- (e.g., Czech and Finnish) and fixed-word-order
languages (e.g., English and French) does not explain cross-lingual variation. Instead, the structure of the word and subword vocabulary strongly predicts the model surprisal. Overall, vocabulary structure emerges as a key driver of computational word-order learnability across
languages.
Read the full abstract ▾
All authors Jonas Mayer Martins, Jaap Jumelet, Viola Priesemann, Lisa Beinborn
Computation and Language
Is Evaluation Awareness Just Format Sensitivity? Limitations of Probe-Based Evidence under Controlled Prompt Structure
Viliana Devbunova
Prior work uses linear probes on
benchmark prompts as evidence of evaluation awareness in large
language models. Because evaluation context is typically entangled with
benchmark format and genre, it is unclear whether probe-based signals reflect context or surface structure. We test whether these signals persist under partial control of
prompt format using a controlled 2x2
dataset and diagnostic rewrites. We find that probes primarily track
benchmark-canonical structure and fail to generalize to free-form
prompts independent of linguistic style. Thus, standard probe-based methodologies do not reliably disentangle evaluation context from structural artifacts, limiting the evidential strength of existing results.
Read the full abstract ▾
Computation and Language
Scalable Prompt Routing via Fine-Grained Latent Task Discovery
Yunyi Zhang, Soji Adeshina, Sheng Guan et al. (+5 more)
Prompt routing dynamically selects the most appropriate large
language model from a pool of candidates for each query, optimizing performance while managing costs. As model pools scale to include dozens of frontier models with narrow performance gaps, existing approaches face significant challenges: manually defined task taxonomies cannot capture fine-grained capability distinctions, while monolithic routers struggle to differentiate subtle differences across diverse tasks. We propose a two-stage routing architecture that addresses these limitations through automated fine-grained task discovery and task-aware quality estimation. Our first stage employs graph-based clustering to discover latent task types and trains a classifier to assign
prompts to discovered tasks. The second stage uses a mixture-of-experts architecture with task-specific prediction heads for specialized quality estimates. At inference, we aggregate predictions from both stages to balance task-level stability with
prompt-specific adaptability. Evaluated on 10
benchmarks with 11 frontier models, our method consistently outperforms existing baselines and surpasses the strongest individual model while incurring less than half its cost.
Read the full abstract ▾
All authors Yunyi Zhang, Soji Adeshina, Sheng Guan, Ashwin Ganesh, Zhen Han, Vassilis N. Ioannidis, Huzefa Rangwala, George Karypis
Computation and Language · Cross-listed from cs.LG
Tomasz Wietrzykowski
Current
transformer language models are trained with uniform computational budgets across all layers, implicitly assuming layer homogeneity. We challenge this assumption through empirical analysis of SmolLM2-135M, a 30-layer, 135M-parameter causal
language model, using five diagnostic metrics: weight predictability (R2), ablation degradation, recovery speed, weight manipulation robustness, and structural analysis. We find profound anatomical heterogeneity: (1) Layer weights follow strong mathematical regularity (R2 = 0.91) with a universal oscillatory delta pattern (correlation ~= -0.50), yet predicted weights cause catastrophic failure due to nonlinear error accumulation. (2) Layer importance spans a 10^7 range, from a critical core (L8-11, up to +63,419% PPL degradation) to anti-layers (L14, L17) whose removal improves performance. (3) Recovery speed correlates with layer importance, indicating differential training requirements. (4) Only weight scaling (alpha = 0.9) preserves model quality among five tested manipulation strategies. (5) Growth
Transformer Training, allocating budget by layer importance, achieves ~54% cost reduction. A proof-of-concept experiment confirms this: 4.7x lower validation loss than uniform training at identical parameter count, while being 13% faster.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
Spectral Tempering for Embedding Compression in Dense Passage Retrieval
Yongkang Li, Panagiotis Eustratiadis and Evangelos Kanoulas
Dimensionality reduction is critical for deploying dense
retrieval systems at scale, yet mainstream post-hoc methods face a fundamental trade-off: principal component analysis (PCA) preserves dominant variance but underutilizes representational capacity, while whitening enforces isotropy at the cost of amplifying noise in the heavy-tailed eigenspectrum of
retrieval embeddings. Intermediate spectral scaling methods unify these extremes by reweighting dimensions with a power coefficient $\gamma$, but treat $\gamma$ as a fixed hyperparameter that requires task-specific tuning. We show that the optimal scaling strength $\gamma$ is not a global constant: it varies systematically with target dimensionality $k$ and is governed by the signal-to-noise ratio (SNR) of the retained subspace. Based on this insight, we propose Spectral Tempering (SpecTemp), a learning-free method that derives an adaptive $\gamma(k)$ directly from the corpus eigenspectrum using local SNR analysis and knee-point normalization, requiring no labeled data or validation-based search. Extensive experiments demonstrate that Spectral Tempering consistently achieves near-oracle performance relative to grid-searched $\gamma^*(k)$ while remaining fully learning-free and model-agnostic. Our code is publicly available at https://github.com/liyongkang123/SpecTemp.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CE
FinTradeBench: A Financial Reasoning Benchmark for LLMs
Yogesh Agrawal, Aniruddha Dutta, Md Mahadi Hasan et al. (+2 more)
Financial analysis requires
reasoning over heterogeneous signals, including company fundamentals from regulatory filings and trading signals derived from
market prices. However, existing financial
benchmarks focus almost exclusively on regulatory filings, neglecting
market dynamics and cross-signal interactions. We introduce FinTradeBench, a
benchmark of 1,400 questions covering NASDAQ-100 stocks over a ten-year window. To holistically evaluate financial
reasoning, FinTradeBench includes questions in three
reasoning categories: fundamentals-focused, trading-signal-focused, and hybrid questions that require cross-signal
reasoning. Our
benchmark starts with 150 expert-authored seed questions, which are then scaled to 1400. To ensure reliable evaluation, we create a Core400 subset in which every item is expert-reviewed. In zero-shot (No-RAG) response generation, we observe that all models perform substantially worse on trading-focused questions compared to fundamental-focused questions. To identify failure causes, we evaluate LLMs under controlled settings including
retrieval-based baselines, tool-assisted computation, oracle conditions, and time-series-specialized pipelines. LLMs' capabilities to do computation and evidence selection contribute most to
reasoning failures. Additionally,
retrieval significantly improves trading-signal
reasoning but does not improve fundamental
reasoning, where identifying relevant signals remains the dominant failure mode. Finally, rather than relying on a small, fine-tuned model for time-series
reasoning, it is better to summarize the time-series data and leverage LLMs'
reasoning capabilities.
Read the full abstract ▾
All authors Yogesh Agrawal, Aniruddha Dutta, Md Mahadi Hasan, Santu Karmaker, Aritra Dutta
Computation and Language
F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World
Ziyin Zhang, Zihan Liao, Hang Yu et al. (+2 more)
We present F2LLM-v2, a new family of general-purpose,
multilingual embedding models in 8 distinct sizes ranging from 80M to 14B. Trained on a newly curated composite of 60 million publicly available high-quality data samples, F2LLM-v2 supports more than 200
languages, with a particular emphasis on previously underserved mid- and low-resource
languages. By integrating a two-stage LLM-based
embedding training pipeline with matryoshka learning, model
pruning, and knowledge
distillation techniques, we present models that are far more efficient than previous LLM-based
embedding models while retaining competitive performances. Extensive evaluations confirm that F2LLM-v2-14B ranks first on 11 MTEB
benchmarks, while the smaller models in the family also set a new state of the art for resource-constrained applications. To facilitate open-source
embedding model research, we release all models, data, code, and intermediate checkpoints.
Read the full abstract ▾
All authors Ziyin Zhang, Zihan Liao, Hang Yu, Peng Di, Rui Wang
Computation and Language · Cross-listed from cs.LG
Online Learning and Equilibrium Computation with Ranking Feedback
Mingyang Liu, Yongshan Chen, Zhiyuan Fan et al. (+3 more)
Online learning in arbitrary, and possibly
adversarial, environments has been extensively studied in sequential decision-making, and it is closely connected to
equilibrium computation in game theory. Most existing online learning
algorithms rely on numeric utility feedback from the environment, which may be unavailable in human-in-the-loop applications and/or may be restricted by
privacy concerns. In this paper, we study an online learning model in which the learner only observes a ranking over a set of proposed actions at each timestep. We consider two ranking mechanisms: rankings induced by the instantaneous utility at the current timestep, and rankings induced by the time-average utility up to the current timestep, under both full-information and bandit feedback settings. Using the standard external-regret metric, we show that sublinear regret is impossible with instantaneous-utility ranking feedback in general. Moreover, when the ranking model is relatively deterministic, i.e., under the Plackett-Luce model with a temperature that is sufficiently small, sublinear regret is also impossible with time-average utility ranking feedback. We then develop new
algorithms that achieve sublinear regret under the additional assumption that the utility sequence has sublinear total variation. Notably, for full-information time-average utility ranking feedback, this additional assumption can be removed. As a consequence, when all players in a normal-form game follow our
algorithms, repeated play yields an approximate coarse correlated
equilibrium. We also demonstrate the effectiveness of our
algorithms in an online large-
language-model routing task.
Read the full abstract ▾
All authors Mingyang Liu, Yongshan Chen, Zhiyuan Fan, Gabriele Farina, Asuman Ozdaglar, Kaiqing Zhang
Computation and Language
Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
Zhuolin Yang, Zihan Liu, Yang Chen et al. (+14 more)
We introduce Nemotron-Cascade 2, an open 30B MoE model with 3B activated parameters that delivers best-in-class
reasoning and strong agentic capabilities. Despite its compact size, its mathematical and coding
reasoning performance approaches that of frontier open models. It is the second open-weight LLM, after DeepSeekV3.2-Speciale-671B-A37B, to achieve Gold Medal-level performance in the 2025 International Mathematical Olympiad (IMO), the International Olympiad in Informatics (IOI), and the ICPC World Finals, demonstrating remarkably high intelligence density with 20x fewer parameters. In contrast to Nemotron-Cascade 1, the key technical advancements are as follows. After SFT on a meticulously curated
dataset, we substantially expand Cascade RL to cover a much broader spectrum of
reasoning and agentic domains. Furthermore, we introduce multi-domain on-policy
distillation from the strongest intermediate teacher models for each domain throughout the Cascade RL process, allowing us to efficiently recover
benchmark regressions and sustain strong performance gains along the way. We release the collection of model checkpoint and training data.
Read the full abstract ▾
All authors Zhuolin Yang, Zihan Liu, Yang Chen, Wenliang Dai, Boxin Wang, Sheng-Chieh Lin, Chankyu Lee, Yangyi Chen, Dongfu Jiang, Jiafan He, Renjie Pi, Grace Lam, Nayeon Lee, Alexander Bukharin, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
Computation and Language · Cross-listed from eess.AS
How Auditory Knowledge in LLM Backbones Shapes Audio Language Models: A Holistic Evaluation
Ke-Han Lu, Szu-Wei Fu, Chao-Han Huck Yang et al. (+13 more)
Large
language models (LLMs) have been widely used as knowledge backbones of Large Audio
Language Models (LALMs), yet how much auditory knowledge they encode through text-only pre-training and how this affects downstream performance remains unclear. We study this gap by comparing different LLMs under two text-only and one audio-grounded setting: (1) direct probing on AKB-2000, a curated
benchmark testing the breadth and depth of auditory knowledge; (2) cascade evaluation, where LLMs reason over text descriptions from an audio captioner; and (3) audio-grounded evaluation, where each LLM is fine-tuned into a Large Audio
Language Model (LALM) with an audio encoder. Our findings reveal that auditory knowledge varies substantially across families, and text-only results are strongly correlated with audio performance. Our work provides empirical grounding for a comprehensive understanding of LLMs in audio research.
Read the full abstract ▾
All authors Ke-Han Lu, Szu-Wei Fu, Chao-Han Huck Yang, Zhehuai Chen, Sung-Feng Huang, Chih-Kai Yang, Yi-Cheng Lin, Chi-Yuan Hsiao, Wenze Ren, En-Pei Hu, Yu-Han Huang, An-Yu Cheng, Cheng-Han Chiang, Yu Tsao, Yu-Chiang Frank Wang, Hung-yi Lee
Computation and Language · Cross-listed from cs.AI
Box Maze: A Process-Control Architecture for Reliable LLM Reasoning
Zou Qiang
Large
language models (LLMs) demonstrate strong generative capabilities but remain vulnerable to
hallucination and unreliable
reasoning under
adversarial prompting. Existing safety approaches – such as reinforcement learning from human feedback (RLHF) and output filtering – primarily operate at the behavioral level and may lack explicit architectural mechanisms for enforcing
reasoning process integrity. This paper proposes the Box Maze framework, a conceptual process-control architecture that decomposes LLM
reasoning into three explicit layers:
memory grounding, structured inference, and boundary enforcement. We introduce preliminary simulation-based evaluation involving progressive boundary erosion scenarios across multiple heterogeneous LLM systems (DeepSeek-V3, Doubao, Qwen). Results from n=50
adversarial scenarios suggest that explicit cognitive control layers may improve consistency in boundary maintenance, with architectural constraints reducing boundary failure rates from approximately 40% (baseline RLHF) to below 1% under
adversarial conditions. While current validation is simulation-based, these preliminary results indicate that process-level control may offer a promising direction for improving reliability in large
language model
reasoning.
Read the full abstract ▾
Computation and Language
Evaluating Counterfactual Strategic Reasoning in Large Language Models
Dimitrios Georgousis, Maria Lymperaiou, Angeliki Dimitriou et al. (+2 more)
We evaluate Large
Language Models (LLMs) in repeated game-theoretic settings to assess whether strategic performance reflects genuine
reasoning or reliance on memorized patterns. We consider two canonical games, Prisoner's Dilemma (PD) and Rock-Paper-Scissors (RPS), upon which we introduce counterfactual variants that alter payoff structures and action labels, breaking familiar
symmetries and dominance relations. Our multi-metric evaluation framework compares default and counterfactual instantiations, showcasing LLM limitations in incentive sensitivity, structural generalization and strategic
reasoning within counterfactual environments.
Read the full abstract ▾
All authors Dimitrios Georgousis, Maria Lymperaiou, Angeliki Dimitriou, Giorgos Filandrianos, Giorgos Stamou
Computation and Language · Cross-listed from cs.RO
Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation
Swagat Padhan, Lakshya Jain, Bhavya Minesh Shah et al. (+3 more)
Robots collaborating with humans must convert natural
language goals into actionable, physically grounded decisions. For example, executing a command such as "go two meters to the right of the fridge" requires grounding semantic references, spatial relations, and metric constraints within a 3D scene. While recent
vision language models (VLMs) demonstrate strong semantic grounding capabilities, they are not explicitly designed to reason about metric constraints in physically defined spaces. In this work, we empirically demonstrate that state-of-the-art VLM-based grounding approaches struggle with complex metric-semantic
language queries. To address this limitation, we propose MAPG (Multi-
Agent Probabilistic Grounding), an agentic framework that decomposes
language queries into structured subcomponents and queries a VLM to ground each component. MAPG then probabilistically composes these grounded outputs to produce metrically consistent, actionable decisions in 3D space. We evaluate MAPG on the HM-EQA
benchmark and show consistent performance improvements over strong baselines. Furthermore, we introduce a new
benchmark, MAPG-Bench, specifically designed to evaluate metric-semantic goal grounding, addressing a gap in existing
language grounding evaluations. We also present a real-world
robot demonstration showing that MAPG transfers beyond simulation when a structured scene representation is available.
Read the full abstract ▾
All authors Swagat Padhan, Lakshya Jain, Bhavya Minesh Shah, Omkar Patil, Thao Nguyen, Nakul Gopalan
Computation and Language
VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Models
Chonghan Liu, Yimin Du, Qi An et al. (+8 more)
Large
language models frequently exhibit suboptimal performance on low resource
languages, primarily due to inefficient subword
segmentation and systemic training data imbalances. In this paper, we propose Variable
Entropy Policy Optimization (VEPO), which leverages Reinforcement Learning with Verifiable Rewards to incorporate deterministic structural constraints into the policy
alignment process. This framework ensures prescribed sequence length, robust format consistency, and rigorous linguistic well formedness, all enforced during training. Central to our approach is a variable
entropy mechanism that enables the model to dynamically calibrate the
equilibrium between literal fidelity and semantic naturalness by modulating the exploration exploitation
manifold. By integrating
entropy tempered advantage estimation with asymmetric clipping, VEPO sustains robust exploration while mitigating policy collapse. Empirical evaluations across 90 FLORES-200, COMET-22, chrF directions demonstrate that VEPO yields substantial improvements in both tokenization efficiency and translation quality, bridging the performance gap for underrepresented
languages.
Read the full abstract ▾
All authors Chonghan Liu, Yimin Du, Qi An, Xin He, Cunqi Zhai, Fei Tan, Weijia Lin, Xiaochun Gong, Yongchao Deng, Shousheng Jia, Xiangzheng Zhang
Computation and Language
Optimal Splitting of Language Models from Mixtures to Specialized Domains
Skyler Seto, Pierre Ablin, Anastasiia Filippova et al. (+4 more)
Language models achieve impressive performance on a variety of knowledge,
language, and
reasoning tasks due to the scale and diversity of pretraining data available. The standard training recipe is a two-stage paradigm: pretraining first on the full corpus of data followed by specialization on a subset of high quality, specialized data from the full corpus. In the multi-domain setting, this involves continued pretraining of multiple models on each specialized domain, referred to as split model training. We propose a method for pretraining multiple models independently over a general pretraining corpus, and determining the optimal compute allocation between pretraining and continued pretraining using scaling laws. Our approach accurately predicts the loss of a model of size N with D pretraining and D' specialization
tokens, and extrapolates to larger model sizes and number of
tokens. Applied to
language model training, our approach improves performance consistently across common sense knowledge and
reasoning benchmarks across different model sizes and compute budgets.
Read the full abstract ▾
All authors Skyler Seto, Pierre Ablin, Anastasiia Filippova, Jiayuan Ye, Louis Bethune, Angelos Katharopoulos, David Grangier
Computation and Language
UGID: Unified Graph Isomorphism for Debiasing Large Language Models
Zikang Ding, Junchi Yao, Junhao Li et al. (+4 more)
Large
language models (LLMs) exhibit pronounced social biases. Output-level or data-optimization–based debiasing methods cannot fully resolve these biases, and many prior works have shown that biases are embedded in internal representations. We propose \underline{U}nified \underline{G}raph \underline{I}somorphism for \underline{D}ebiasing large
language models (\textit{UGID}), an internal-representation–level debiasing framework for large
language models that models the
Transformer as a structured computational graph, where
attention mechanisms define the routing edges of the graph and hidden states define the graph nodes. Specifically, debiasing is formulated as enforcing invariance of the graph structure across counterfactual inputs, with differences allowed only on sensitive attributes. \textit{UGID} jointly constrains
attention routing and hidden representations in bias-sensitive regions, effectively preventing bias migration across architectural components. To achieve effective behavioral
alignment without degrading general capabilities, we introduce a log-space constraint on sensitive logits and a selective anchor-based objective to preserve definitional semantics. Extensive experiments on large
language models demonstrate that \textit{UGID} effectively reduces bias under both in-distribution and out-of-distribution settings, significantly reduces internal structural discrepancies, and preserves model safety and utility.
Read the full abstract ▾
All authors Zikang Ding, Junchi Yao, Junhao Li, Yi Zhang, Wenbo Jiang, Hongbo Liu, Lijie Hu
Computation and Language · Cross-listed from cs.AI
How Uncertainty Estimation Scales with Sampling in Reasoning Models
Maksym Del, Markus Kängsepp, Marharyta Domnich et al. (+4 more)
Uncertainty estimation is critical for deploying
reasoning language models, yet remains poorly understood under extended chain-of-thought
reasoning. We study parallel sampling as a fully black-box approach using verbalized confidence and self-consistency. Across three
reasoning models and 17 tasks spanning mathematics, STEM, and humanities, we characterize how these signals scale. Both self-consistency and verbalized confidence scale in
reasoning models, but self-consistency exhibits lower initial discrimination and lags behind verbalized confidence under moderate sampling. Most uncertainty gains, however, arise from signal combination: with just two samples, a hybrid
estimator improves AUROC by up to $+12$ on average and already outperforms either signal alone even when scaled to much larger budgets, after which returns diminish. These effects are domain-dependent: in mathematics, the native domain of RLVR-style post-training,
reasoning models achieve higher uncertainty quality and exhibit both stronger complementarity and faster scaling than in STEM or humanities.
Read the full abstract ▾
All authors Maksym Del, Markus Kängsepp, Marharyta Domnich, Ardi Tampuu, Lisa Yankovskaya, Meelis Kull, Mark Fishel
Computation and Language
DaPT: A Dual-Path Framework for Multilingual Multi-hop Question Answering
Yilin Wang, Yuchun Fan, Jiaoyang Li et al. (+5 more)
Retrieval-augmented generation (RAG) systems have made significant progress in solving complex multi-hop question answering (QA) tasks in the English scenario. However, RAG systems inevitably face the application scenario of retrieving across
multilingual corpora and queries, leaving several open challenges. The first one involves the absence of
benchmarks that assess RAG systems' capabilities under the
multilingual multi-hop (MM-hop) QA setting. The second centers on the overreliance on LLMs' strong semantic understanding in English, which diminishes effectiveness in
multilingual scenarios. To address these challenges, we first construct
multilingual multi-hop QA
benchmarks by translating English-only
benchmarks into five
languages, and then we propose DaPT, a novel
multilingual RAG framework. DaPT generates sub-question graphs in parallel for both the source-
language query and its English translation counterpart, then merges them before employing a bilingual
retrieval-and-answer strategy to sequentially solve sub-questions. Our experimental results demonstrate that advanced RAG systems suffer from a significant performance imbalance in
multilingual scenarios. Furthermore, our proposed method consistently yields more accurate and concise answers compared to the baselines, significantly enhancing RAG performance on this task. For instance, on the most challenging MuSiQue
benchmark, DaPT achieves a relative improvement of 18.3% in average EM score over the strongest baseline.
Read the full abstract ▾
All authors Yilin Wang, Yuchun Fan, Jiaoyang Li, Ziming Zhu, Yongyu Mu, Qiaozhi He, Tong Xiao, Jingbo Zhu
Computation and Language · Cross-listed from cs.CV
SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cues
Carlos Hinojosa, Clemens Grange and Bernard Ghanem
Vision-
language models (VLMs) are increasingly deployed in real-world and
embodied settings where safety decisions depend on visual context. However, it remains unclear which visual evidence drives these judgments. We study whether
multimodal safety behavior in VLMs can be steered by simple semantic cues. We introduce a semantic steering framework that applies controlled textual, visual, and cognitive interventions without changing the underlying scene content. To evaluate these effects, we propose SAVeS, a
benchmark for situational safety under semantic cues, together with an evaluation protocol that separates behavioral refusal, grounded safety
reasoning, and false refusals. Experiments across multiple VLMs and an additional state-of-the-art
benchmark show that safety decisions are highly sensitive to semantic cues, indicating reliance on learned visual-linguistic associations rather than grounded visual understanding. We further demonstrate that automated steering pipelines can exploit these mechanisms, highlighting a potential vulnerability in
multimodal safety systems.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Assessing the Effect of Cross-Domain Mapping on Creativity in Humans and Large Language Models
Qiawen Ella Liu, Marina Dubova, Henry Conklin et al. (+2 more)
Creative ideas often arise by associating remote concepts. Can random associations reliably increase originality, and do they help humans and large
language models (LLMs) in the same way? We asked human participants and seven LLMs to design products by drawing inspiration from a random source or addressing an unmet user need. Humans reliably benefited from cross-domain mappings, while LLMs generated more original ideas than humans but showed no overall benefit from the intervention, though this changed with semantic distance. More distant source-target pairings produced more original ideas in both humans and LLMs. Humans benefited at nearly any distance, while only the highest-rated LLMs benefited when the source was sufficiently remote. Humans and LLMs also used sources differently: humans tended to transfer surface features, while LLMs transferred structural and functional properties. These findings reveal the generative role of remote associations and systematic differences in how humans and AI respond to the same creativity intervention.
Read the full abstract ▾
All authors Qiawen Ella Liu, Marina Dubova, Henry Conklin, Takumi Harada, Thomas L. Griffiths
Computation and Language
A Dataset and Resources for Identifying Patient Health Literacy Information from Clinical Notes
Madeline Bittner, Dina Demner-Fushman, Yasmeen Shabazz et al. (+6 more)
Health literacy is a critical determinant of patient outcomes, yet current screening tools are not always feasible and differ considerably in the number of items, question format, and dimensions of health literacy they capture, making documentation in structured electronic health records difficult to achieve. Automated detection from unstructured clinical notes offers a promising alternative, as these notes often contain richer, more contextual health literacy information, but progress has been limited by the lack of annotated resources. We introduce HEALIX, the first publicly available annotated health literacy
dataset derived from real clinical notes, curated through a combination of social worker note sampling, keyword-based filtering, and LLM-based active learning. HEALIX contains 589 notes across 9 note types, annotated with three health literacy labels: low, normal, and high. To demonstrate its utility, we benchmarked zero-shot and few-shot prompting strategies across four open source large
language models (LLMs).
Read the full abstract ▾
All authors Madeline Bittner, Dina Demner-Fushman, Yasmeen Shabazz, Davis Bartels, Dukyong Yoon, Brad Quitadamo, Rajiv Menghrajani, Leo Celi, Sarvesh Soni
Computation and Language
Large Language Models provide support for the parallelogram theory of analogy
Qiawen Ella Liu, Raja Marjieh, Jian-Qiao Zhu et al. (+2 more)
Four-term word analogies (A:B::C:D) are classically modeled geometrically as parallelograms: adding the vector B-A+C produces D. Recent work suggests that this model poorly captures how humans produce analogies, with simple local-similarity heuristics often providing a better account (Peterson et al., 2020). But does the parallelogram model fail because it is a bad model of analogical relations, or because people are not very good at generating relation-preserving analogies? We compared human and large
language model (LLM) analogy completions on the set of problems from Peterson et al. (2020). We find that LLM-generated analogies are reliably judged as better than human-generated ones, and are also more consistent with parallelograms in a distributional
embedding space. Crucially, we show that the improvement over human analogies is driven by greater parallelogram
alignment and reduced reliance on accessible words rather than enhanced sensitivity to local similarity. Finally, fine-tuning GloVe to better satisfy the parallelogram constraint makes the model's top-ranked candidates more likely to be the completions humans and LLMs actually produced, and improves its prediction of human ratings. Overall, these results provide support for the parallelogram model of word analogies.
Read the full abstract ▾
All authors Qiawen Ella Liu, Raja Marjieh, Jian-Qiao Zhu, Adele E. Goldberg, Thomas L. Griffiths
Computation and Language
MoRI: Learning Motivation-Grounded Reasoning for Scientific Ideation in Large Language Models
Chenyang Gu, Jiahao Cheng, Meicong Zhang et al. (+3 more)
Scientific ideation aims to propose novel solutions within a given scientific context. Existing LLM-based agentic approaches emulate human research workflows, yet inadequately model scientific
reasoning, resulting in surface-level conceptual recombinations that lack technical depth and scientific grounding. To address this issue, we propose MoRI (Motivation-grounded
Reasoning for Scientific Ideation), a framework that enables LLMs to explicitly learn the
reasoning process from research motivations to methodologies. The base LLM is initialized via supervised fine-tuning to generate a research motivation from a given context, and is subsequently trained under a composite reinforcement learning reward that approximates scientific rigor: (1)
entropy-aware information gain encourages the model to uncover and elaborate high-complexity technical details grounded in ground-truth methodologies, and (2) contrastive semantic gain constrains the
reasoning trajectory to remain conceptually aligned with scientifically valid solutions. Empirical results show that MoRI consistently outperforms strong commercial LLMs and complex agentic baselines across multiple dimensions, including novelty, technical rigor, and feasibility. The code is available on \href{https://github.com/ECNU-Text-Computing/IdeaGeneration}{GitHub}.
Read the full abstract ▾
All authors Chenyang Gu, Jiahao Cheng, Meicong Zhang, Pujun Zheng, Jinquan Zheng, Guoxiu He
Computation and Language
What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?
Gagan Bhatia, Ahmad Muhammad Isa, Maxime Peyrard et al. (+1 more)
We present MultiTempBench, a
multilingual temporal
reasoning benchmark spanning three tasks, date arithmetic, time zone conversion, and temporal relation extraction across five
languages (English, German, Chinese, Arabic, and Hausa) and multiple calendar conventions (Gregorian, Hijri, and Chinese Lunar). MultiTempBench contains $15,000$ examples built by translating $750$ curated English questions and expanding each into controlled date-format variants. We evaluate 20 LLMs and introduce the
multilingual Date Fragmentation Ratio (mDFR), calibrated with human severity ratings, together with
geometric-probing analyses of internal temporal representations. We find tokenisation quality of temporal artefacts is a resource-dependent bottleneck: in low-resource
languages and rarer calendar formats, fragmentation disrupts Year/Month/Day separation and accuracy collapses, while high-resource settings are often robust to digit-level splitting. Beyond tokenisation, crossed mixed-effects regression shows that temporal linearity is the strongest predictor of temporal
reasoning in high-resource
languages, whereas fragmentation is the stronger predictor in low-resource
languages. Code is available at: https://github.com/gagan3012/mtb
Read the full abstract ▾
All authors Gagan Bhatia, Ahmad Muhammad Isa, Maxime Peyrard, Wei Zhao
Computation and Language
Hypothesis-Conditioned Query Rewriting for Decision-Useful Retrieval
Hangeol Chang, Changsun Lee, Seungjoon Rho et al. (+2 more)
Retrieval-Augmented Generation (RAG) improves Large
Language Models (LLMs) by grounding generation in external, non-parametric knowledge. However, when a task requires choosing among competing options, simply grounding generation in broadly relevant context is often insufficient to drive the final decision. Existing RAG methods typically rely on a single initial query, which often favors topical relevance over decision-relevant evidence, and therefore retrieves background information that can fail to discriminate among answer options. To address this issue, here we propose Hypothesis-Conditioned Query Rewriting (HCQR), a training-free pre-
retrieval framework that reorients RAG from topic-oriented
retrieval to evidence-oriented
retrieval. HCQR first derives a lightweight working hypothesis from the input question and candidate options, and then rewrites
retrieval into three targeted queries that seek evidence to: (1) support the hypothesis, (2) distinguish it from competing alternatives, and (3) verify salient clues in the question. This approach enables context
retrieval that is more directly aligned with answer selection, allowing the generator to confirm or overturn the initial hypothesis based on the retrieved evidence. Experiments on MedQA and MMLU-Med show that HCQR consistently outperforms single-query RAG and re-rank/filter baselines, improving average accuracy over Simple RAG by 5.9 and 3.6 points, respectively. Code is available at https://anonymous.4open.science/r/HCQR-1C2E.
Read the full abstract ▾
All authors Hangeol Chang, Changsun Lee, Seungjoon Rho, Junho Yeo, Jong Chul Ye
Computation and Language
RADIUS: Ranking, Distribution, and Significance - A Comprehensive Alignment Suite for Survey Simulation
Weronika Łajewska, Paul Missault, George Davidson et al. (+1 more)
Simulating survey responses with LLMs offers a scalable alternative to costly human data collection, yet evaluation methods remain ad hoc and inconsistent. Current metrics, typically borrowed from adjacent domains, focus on accuracy or distributional similarity while overlooking ranking
alignment-whether simulations preserve the relative preference orderings observed in human populations. A simulator may achieve low distributional divergence yet fail to identify the option humans most prefer, a critical gap for decision-making applications. We propose RADIUS, a two-dimensional
alignment suite that evaluates survey simulation along complementary axes: 1) RA
alignment and 2) DIstribUtion
alignment, accompanied by statistical Significance testing. Experiments across multiple social surveys and persona sources demonstrate that RADIUS exposes limitations of single-metric evaluation and offers stronger discriminative power. On an independently reproduced simulator, we further show that distributional improvement often coincides with misidentifying the option humans most prefer - a decision-relevant error RADIUS surfaces but distributional metrics miss. We open-source RADIUS to support reproducible and comparable assessment of survey simulators.
Read the full abstract ▾
All authors Weronika Łajewska, Paul Missault, George Davidson, Saab Mansour
Computation and Language · Cross-listed from cs.CY
A conceptual framework for ideology in online discourse beyond the left and right
Kenneth Joseph, Kim Williams and David Lazer
Computational social science (CSS) has largely operationalized ideology along a single left/right partisan axis when studying online discourse. This approach obscures how people interpret and engage with more specific ideological formations related to race,
climate, gender, and other domains. We introduce a framework that instead conceptualizes ideology as a multi-level socio-cognitive concept network and then explain how this conceptual model of ideology can be linked to the study of online discourse. In doing so, our framework clarifies how ideology manifests in discourse alongside related social processes such as framing, and provides an argument for better understanding of when and why we might study multiple concepts, such as values and beliefs, together in one analysis. More broadly, it bridges methods used to study online discourse with ideology theory, enabling richer analyses of social discourse that benefit both fields. Although primarily theoretical, we provide a concrete operationalization of the framework on a sample of data, allowing us to show one example of how it can be used on real data.
Read the full abstract ▾
Computation and Language
Entropy trajectory shape predicts LLM reasoning reliability: A diagnostic study of uncertainty dynamics in chain-of-thought
Xinghao Zhao
Understanding uncertainty in chain-of-thought
reasoning is critical for reliable deployment of large
language models. In this work, we propose a simple yet effective diagnostic approach based on trajectory shape rather than scalar magnitude. We show that this signal is practical, interpretable, and inexpensive to obtain in black-box settings, while remaining robust across models and
datasets. Through extensive ablations and cross-domain replications, we demonstrate its utility for selective prediction and triage. Our findings offer a generalizable insight into uncertainty dynamics in
reasoning tasks, with particular focus on numeric and discrete-answer settings.
Read the full abstract ▾