Computation and Language · Cross-listed from cs.HC
Taxonomy of User Needs and Actions
Renee Shelby, Fernando Diaz and Vinodkumar Prabhakaran
The growing ubiquity of conversational AI highlights the need for frameworks that capture not only users' instrumental goals but also the situated, adaptive, and social practices through which they achieve them. Existing taxonomies of conversational behavior either overgeneralize, remain domain-specific, or reduce interactions to narrow dialogue functions. To address this gap, we introduce the Taxonomy of User Needs and Actions (TUNA), an empirically grounded framework developed through iterative qualitative analysis of 1193 human-AI conversations, supplemented by theoretical review and validation across diverse contexts. TUNA organizes user actions into a three-level hierarchy encompassing behaviors associated with information seeking, synthesis, procedural guidance, content creation, social interaction, and meta-conversation. By centering user agency and appropriation practices, TUNA enables multi-scale evaluation, supports policy harmonization across products, and provides a backbone for layering domain-specific taxonomies. This work contributes a systematic vocabulary for describing AI use, advancing both scholarly understanding and practical design of safer, more responsive, and more accountable conversational systems.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Influence Functions for Efficient Data Selection in Reasoning
Prateek Humane, Paolo Cudrano, Daniel Z. Kaplan et al. (+3 more)
Fine-tuning large
language models (LLMs) on chain-of-thought (CoT) data shows that a small amount of high-quality data can outperform massive
datasets. Yet, what constitutes "quality" remains ill-defined. Existing
reasoning methods rely on indirect heuristics such as problem difficulty or trace length, while instruction-tuning has explored a broader range of automated selection strategies, but rarely in the context of
reasoning. We propose to define
reasoning data quality using influence functions, which measure the causal effect of individual CoT examples on downstream accuracy, and introduce influence-based
pruning, which consistently outperforms perplexity and
embedding-based baselines on math
reasoning within a model family.
Read the full abstract ▾
All authors Prateek Humane, Paolo Cudrano, Daniel Z. Kaplan, Matteo Matteucci, Supriyo Chakraborty, Irina Rish
Computation and Language
Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Models
Gagan Bhatia, Somayajulu G Sripada, Kevin Allan et al. (+1 more)
Hallucinations in large
language models (LLMs) produce fluent continuations that are not supported by the
prompt, especially under minimal contextual cues and ambiguity. We introduce Distributional Semantics Tracing (DST), a model-native method that builds layer-wise semantic maps at the answer position by decoding residual-stream states through the unembedding, selecting a compact top-$K$ concept set, and estimating directed concept-to-concept support via lightweight causal tracing. Using these traces, we test a representation-level hypothesis:
hallucinations arise from correlation-driven representational drift across depth, where the residual stream is pulled toward a locally coherent but context-inconsistent concept neighborhood reinforced by training co-occurrences. On Racing Thoughts
dataset, DST yields more faithful explanations than attribution, probing, and intervention baselines under an LLM-judge protocol, and the resulting Contextual
Alignment Score (CAS) strongly predicts failures, supporting this drift hypothesis.
Read the full abstract ▾
All authors Gagan Bhatia, Somayajulu G Sripada, Kevin Allan, Jacobo Azcona
Computation and Language
Muyu He, Muhammad Ali Shafique, Anand Kumar et al. (+2 more)
Distilling the thinking traces of a Large
Language Model (LLM) with
reasoning capabilities into a smaller model has been proven effective. Yet, there is a scarcity of work done on how model performances scale with the quantity of
distillation data. In this work, we study the scaling trend of distilling competitive coding skills on two small non-
reasoning LLMs. We validate the hypothesis that there is a $\textit{valley of code reasoning}$: downstream performance on competitive coding first drops as data quantity increases, then it steadily increases in a sharper-than-log-linear fashion. Having identified the trend, we further fine-tune the models at two different
distillation stages on the same data to ground conclusions on their respective learning phases. We learn that across stages in the low and medium-low data regimes, small models benefit significantly from easier coding questions than from harder ones. We also find that, surprisingly, the correctness of outputs in training data makes no difference to
distillation outcomes. Our work represents a step forward in understanding the training dynamics of code
reasoning distillation outside intuition
Read the full abstract ▾
All authors Muyu He, Muhammad Ali Shafique, Anand Kumar, Tsach Mackey, Nazneen Rajani
Computation and Language · Cross-listed from cs.LG
The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM Objectives
Matthieu Bou, Nyal Patel, Arjun Jagota et al. (+2 more)
The objectives that Large
Language Models (LLMs) implicitly optimize remain dangerously opaque, making trustworthy
alignment and
auditing a grand challenge. While Inverse Reinforcement Learning (IRL) can infer reward functions from behaviour, existing approaches either produce a single, overconfident reward estimate or fail to address the fundamental ambiguity of the task (non-identifiability). This paper introduces a principled
auditing framework that re-frames reward inference from a simple estimation task to a comprehensive process for verification. Our framework leverages
Bayesian IRL to not only recover a distribution over objectives but to enable three critical audit capabilities: (i) Quantifying and systematically reducing non-identifiability by demonstrating posterior contraction over sequential rounds of evidence; (ii) Providing actionable, uncertainty-aware diagnostics that expose spurious shortcuts and identify out-of-distribution
prompts where the inferred objective cannot be trusted; and (iii) Validating policy-level utility by showing that the refined, low-uncertainty reward can be used directly in RLHF to achieve training dynamics and toxicity reductions comparable to the ground-truth
alignment process. Empirically, our framework successfully audits a detoxified LLM, yielding a well-calibrated and interpretable objective that strengthens
alignment guarantees. Overall, this work provides a practical toolkit for auditors, safety teams, and regulators to verify what LLMs are truly trying to achieve, moving us toward more trustworthy and accountable AI.
Read the full abstract ▾
All authors Matthieu Bou, Nyal Patel, Arjun Jagota, Satyapriya Krishna, Sonali Parbhoo
Computation and Language · Cross-listed from cs.LG
Learning from Failures: Understanding LLM Alignment through Failure-Aware Inverse RL
Nyal Patel, Matthieu Bou, Arjun Jagota et al. (+2 more)
Reinforcement Learning from Human Feedback (RLHF) aligns Large
Language Models (LLMs) with human preferences, yet the underlying reward signals they internalize remain hidden, posing a critical challenge for interpretability and safety. Existing approaches attempt to extract these latent incentives using Inverse Reinforcement Learning (IRL), but treat all preference pairs equally, often overlooking the most informative signals: those examples the extracted reward model misclassifies or assigns nearly equal scores, which we term failures. We introduce a novel failure-aware IRL
algorithm that focuses on misclassified or difficult examples to recover the latent rewards defining model behaviors. By learning from these failures, our failure-aware IRL extracts reward functions that better reflect the true objectives behind RLHF. We demonstrate that failure-aware IRL outperforms existing IRL baselines across multiple metrics when applied to LLM detoxification, without requiring external classifiers or supervision. Crucially, failure-aware IRL yields rewards that better capture the true incentives learned during RLHF, enabling more effective re-RLHF training than standard IRL. This establishes failure-aware IRL as a robust, scalable method for
auditing model
alignment and reducing ambiguity in the IRL process.
Read the full abstract ▾
All authors Nyal Patel, Matthieu Bou, Arjun Jagota, Satyapriya Krishna, Sonali Parbhoo
Computation and Language
Spectrum Tuning: Post-Training for Distributional Coverage and In-Context Steerability
Taylor Sorensen, Benjamin Newman, Jared Moore et al. (+5 more)
Language model post-training has enhanced instruction-following and performance on many downstream tasks, but also comes with an often-overlooked cost on tasks with many possible valid answers. On many tasks such as creative writing, synthetic data generation, or steering to diverse preferences, models must cover an entire distribution of outputs, rather than a single correct answer. We characterize three desiderata for conditional distributional modeling: in-context steerability, valid output space coverage, and distributional
alignment, and document across three model families how current post-training can reduce these properties. In particular, we disambiguate between two kinds of in-context learning: ICL for eliciting existing underlying knowledge or capabilities, and in-context steerability, where a model must use in-context information to override its priors and steer to a novel data generating distribution. To better evaluate and improve these desiderata, we introduce Spectrum Suite, a large-scale resource compiled from >40 data sources and spanning >90 tasks requiring models to steer to and match diverse distributions ranging from varied human preferences to numerical distributions and more. We find that while current post-training techniques elicit underlying capabilities and knowledge, they hurt models' ability to flexibly steer in-context. To mitigate these issues, we propose Spectrum Tuning, a post-training method using Spectrum Suite to improve steerability and distributional coverage. We find that Spectrum Tuning often improves over pretrained and typical instruction-tuned models, enhancing steerability, spanning more of the output space, and improving distributional
alignment on held-out
datasets.
Read the full abstract ▾
All authors Taylor Sorensen, Benjamin Newman, Jared Moore, Chan Park, Jillian Fisher, Niloofar Mireshghallah, Liwei Jiang, Yejin Choi
Computation and Language
When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RL
Jiakang Wang, Runze Liu, Qingpeng Cai et al. (+7 more)
Reinforcement learning (RL) has shown great promise in large
language models (LLMs) post-training, which typically rely on
token-level clipping to maintain stability during optimization. Despite the empirical success of GRPO-style methods, we identify a fundamental and previously overlooked challenge in this popular Outcome-Supervised RL (OSRL) paradigm. We reveal that in OSRL, where advantages are shared across
tokens within a response, importance sampling (IS) ratios deviate from their traditional purpose of distribution correction as in classic RL, which become
token-level weights that allocate the shared advantage signal across
tokens. We show that this hidden role shift induces a critical mismatch for positive-advantage
tokens, leading to unbalanced
token weighting between positive and negative
tokens. Specifically, it suppresses the update of underrepresented
tokens that are lagging behind, while over-amplifying already high-probability
tokens. This mismatch results in rich-get-richer dynamics that over-reinforce confident
tokens, weaken catch-up learning that drive
entropy collapse, excessive repetition, and premature convergence. To address this, we propose Asymmetric Importance Sampling Policy Optimization (ASPO), a simple yet effective strategy that reverses the ratio-induced weighting of positive-advantage
tokens, while stabilizing extreme updates and maintaining gradient flow. This mismatch correction aligns their update direction with the learning dynamics of negative ones. Comprehensive experiments across math
reasoning and coding
benchmarks demonstrate that ASPO significantly mitigates
entropy collapse, improves training stability, and enhances performance over strong GRPO-based baselines. Our analysis provides new insights into the role of
token-level weighting in OSRL and highlights the critical importance of correcting ratio-induced weighting in LLM RL.
Read the full abstract ▾
All authors Jiakang Wang, Runze Liu, Qingpeng Cai, Lei Lin, Wenping Hu, Xiu Li, Fuzheng Zhang, Guorui Zhou, Kun Gai, Ling Pan
Computation and Language · Cross-listed from cs.AI
MixReasoning: Switching Modes to Think
Haiquan Lu, Gongfan Fang, Xinyin Ma et al. (+2 more)
Reasoning models enhance performance by tackling problems in a step-by-step manner, decomposing them into sub-problems and exploring long chains of thought before producing an answer. However, applying extended
reasoning to every step introduces substantial redundancy, as sub-problems vary widely in difficulty and complexity: a small number of pivotal steps are genuinely challenging and decisive for the final answer, while many others only involve straightforward revisions or simple computations. Therefore, a natural idea is to endow
reasoning models with the ability to adaptively respond to this variation, rather than treating all steps with the same level of elaboration. To this end, we propose MixReasoning, a framework that dynamically adjusts the depth of
reasoning within a single response. The resulting chain of thought then becomes a mixture of detailed
reasoning on difficult steps and concise inference on simpler ones. Experiments on GSM8K, MATH-500, and AIME show that MixReasoning shortens
reasoning length and substantially improves efficiency without compromising accuracy.
Read the full abstract ▾
All authors Haiquan Lu, Gongfan Fang, Xinyin Ma, Qi Li, Xinchao Wang
Computation and Language
A Large-Scale Chinese Knowledge Graph-Text Alignment Dataset for Benchmarking Knowledge-Grounded LLMs
Chengwei Wu, Xingrui Zhuo, Mingyang Gao et al. (+3 more)
Reliable evaluation of knowledge-grounded Large
Language Models (LLMs) in Chinese requires resources that explicitly align Chinese-
language text with verifiable Knowledge Graph (KG) facts. Yet existing Chinese
benchmarks primarily assess general
language understanding and offer limited support for structured
reasoning under Chinese-specific linguistic phenomena. We introduce the Chinese Data-Text Pair (CDTP), a large-scale Chinese KG-text
alignment dataset comprising more than 7 million aligned instances across four broad domains. Each instance pairs a Chinese-
language text with one or more textually supported KG triples, totaling 15 million triples. A multi-stage construction pipeline combining
alignment filtering, manual verification, and external evidence validation improves semantic consistency and factual reliability. CDTP supports Knowledge Graph Completion (KGC), Question Answering (QA), and Triple-to-Text Generation (T2T). Across all three tasks, the
benchmark design accounts for Chinese-specific phenomena, including polysemy, word-
segmentation ambiguity, and context-dependent entity interpretation, enabling the evaluation of structured
reasoning, ambiguity-aware factual understanding, and knowledge-grounded generation. Experiments with diverse open-source and proprietary LLMs show that model scale alone does not guarantee reliable performance on these Chinese knowledge-intensive tasks, whereas supervised fine-tuning on CDTP consistently improves in-domain performance and out-of-distribution robustness. The publicly accessible
dataset, code, and evaluation protocols provide a reusable resource for developing and evaluating knowledge-grounded LLMs in Chinese.
Read the full abstract ▾
All authors Chengwei Wu, Xingrui Zhuo, Mingyang Gao, Xinghe Cheng, Zhichao Yan, Jiapu Wang
Computation and Language
Evaluating The Impact of Stimulus Quality in Investigations of LLM Language Performance
Timothy Pistotti, Jason Brown and Michael Witbrock
Recent studies employing Large
Language Models (LLMs) to test the Argument from the Poverty of the Stimulus (APS) have yielded contrasting results across syntactic phenomena. This paper investigates the hypothesis that characteristics of the stimuli used in recent studies, including lexical ambiguities and structural complexities, may confound model performance. A methodology is proposed for re-evaluating LLM competence on syntactic prediction, focusing on GPT-2. This involves: 1) establishing a baseline on previously used (both filtered and unfiltered) stimuli, and 2) generating a new, refined
dataset using a state-of-the-art (SOTA) generative LLM (Gemini 2.5 Pro Preview) guided by linguistically-informed templates designed to mitigate identified confounds. Our preliminary findings indicate that GPT-2 demonstrates notably improved performance on these refined PG stimuli compared to baselines, suggesting that stimulus quality significantly influences outcomes in surprisal-based evaluations of LLM syntactic competency.
Read the full abstract ▾
Computation and Language
MASA: Rethinking the Representational Bottleneck in LoRA with Multi-A Shared Adaptation
Qin Dong, Yuntian Tang, Heming Jia et al. (+7 more)
Low-Rank Adaptation (LoRA) has emerged as a dominant method in Parameter-Efficient Fine-Tuning (PEFT) for large
language models, which augments the
transformer layer with one down-projection $A$ and one up-projection $B$. However, LoRA's reliance on a single down-projection
matrix ($A$) creates a representational bottleneck, as this solitary feature extractor is inherently insufficient for capturing the diverse signals required by complex tasks. This motivates our architectural shift to focus on enriching the feature adaptation to improve the downstream task adaptation ability. We propose MASA (Multi-$A$ Shared Adaptation), an architecture that implements a multi-$A$, single-$B$ structure where the multi-$A$ expert ensemble is asymmetrically shared across layers to ensure parameter efficiency. In MASA, these specialized experts capture diverse features, which are then integrated by a single, layer-specific $B$-
matrix. The effectiveness and versatility of our method are validated through a comprehensive suite of experiments spanning multi-domain generalization, single-domain specialization, and multi-task
reasoning. For example, on the MMLU
benchmark, MASA achieves an average accuracy of 59.62%, outperforming the standard LoRA by 1.08 points (a relative improvement of 1.84%) with comparable learnable parameters of 0.52%.
Read the full abstract ▾
All authors Qin Dong, Yuntian Tang, Heming Jia, Yunhang Shen, Bohan Jia, Wenxuan Huang, Lianyue Zhang, Jiao Xie, Shaohui Lin, Rongrong Ji
Computation and Language · Cross-listed from cs.AI
Deterministic Legal Agents: A Canonical Primitive API for Auditable Reasoning over Temporal Knowledge Graphs
Hudson de Martim
In high-stakes legal domains,
retrieval must preserve not only semantic relevance, but also the hierarchy, temporality, and causal provenance of legal norms. Standard
Retrieval-Augmented Generation (RAG), based mainly on semantic similarity over text fragments, cannot reliably provide this level of control. Prior work on SAT-Graph RAG addressed the representation problem by modeling legal materials as structure-aware temporal knowledge graphs. This paper addresses the next problem: how an LLM-based
reasoning agent can interact with such a graph without reintroducing unreliable
retrieval behavior. We specify the SAT-Graph API, a canonical primitive interface for auditable
reasoning over temporal knowledge graphs, developed and illustrated in the legal domain. The API exposes typed, atomic, and composable primitives that mediate between a probabilistic
language model and a deterministic symbolic substrate. Its design follows Probability Isolation: uncertainty is confined to intent translation, semantic anchoring, and final narrative synthesis, while structural, temporal, and causal graph traversals are executed through deterministic operations over canonical anchors. The interface shifts legal RAG from single-shot Retrieve-then-Generate to active Reason-Act-Observe. An
agent decomposes a legal question into an explicit execution plan, invokes primitives for point-in-time
retrieval, context reconstruction, provenance tracing, and impact analysis, and produces an answer grounded in an auditable log of graph operations. The result is a formal architectural specification, not an empirical
benchmark: a secure interaction protocol that decouples legal knowledge representation from agentic
reasoning. Although illustrated in law, the primitive model is domain-portable to other temporally versioned, provenance-sensitive, and authority-governed knowledge bases.
Read the full abstract ▾
Computation and Language
Exploring Gaps in the APS: Direct Minimal Pair Analysis in LLM Syntactic Assessments
Timothy Pistotti, Jason Brown and Michael Witbrock
Recent studies probing the Argument from the Poverty of the Stimulus (APS) have applied Large
Language Models (LLMs) to test the learnability of complex syntax through surprisal-based metrics. However, divergent conclusions raise questions concerning the insights these metrics offer. While Wilcox et al. (2024) used direct minimal pair comparisons (the "wh-effect") to demonstrate that models successfully generalise knowledge of filler-gap dependencies, Lan et al. (2024) used a Difference-in-Differences (DiD) metric and found that models largely fail on parasitic gaps (PGs). This paper argues that the direct minimal pair approach offers greater diagnostic transparency. We demonstrate this by generating a full 8-permutation paradigm of refined PG stimuli and evaluating the GPT-2 model used in previous studies with a systematic Wilcox-style wh-effect analysis. Our results show that GPT-2 succeeds across all four tested conditions, indicating robust knowledge of filler-gap licensing principles even in complex PG environments. This finding, which contrasts with the more ambiguous results from DiD-style metrics, suggests that the choice of evaluation metric is critical for assessing an LLM's syntactic competence.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs
Xueyan Li, Guinan Su, Mrinmaya Sachan et al. (+1 more)
Large
Language Models (LLMs) are increasingly applied to complex tasks that require extended
reasoning. In such settings, models often benefit from diverse chains-of-thought to arrive at multiple candidate solutions. This requires two competing objectives: to inject enough stochasticity to explore multiple
reasoning chains, and to ensure sufficient accuracy and quality in each path. Existing works pursue the first objective by increasing exploration at highly uncertain steps with higher temperature or larger candidate
token sets, while others improve reliability by rejecting samples with low confidence post-generation, implying that low confidence correlates with low answer quality. These two lines of thought are in conflict, as they conflate different sources of uncertainty. To resolve this, we argue that the decoding rule should be calibrated by correctness, not confidence alone. We should sample from
tokens with higher estimated correctness, and reduce sampling where expected correctness is low. We propose simple strategies that achieve this goal: Greedy-Threshold makes sampling greedy at very low confidence steps. Calibrated-TopK and Calibrated-epsilon set truncation threshold based on estimated rank-wise correctness. Together, our findings challenge prevailing heuristics about decoding under uncertainty and show gains across math and general
reasoning benchmarks.
Read the full abstract ▾
All authors Xueyan Li, Guinan Su, Mrinmaya Sachan, Jonas Geiping
Computation and Language
LexiCon: a Benchmark for Planning under Temporal Constraints in Natural Language
Periklis Mantenoglou, Rishi Hazra, Pedro Zuidberg Dos Martires et al. (+1 more)
Owing to their
reasoning capabilities, large
language models (LLMs) have been evaluated on planning tasks described in natural
language. However, LLMs have largely been tested on planning domains without constraints. In order to deploy them in real-world settings where adherence to constraints, in particular safety constraints, is critical, we need to evaluate their performance on constrained planning tasks. We introduce LexiCon – a natural
language-based (Lexi) constrained (Con) planning
benchmark, consisting of a suite of environments, that can be used to evaluate the planning capabilities of LLMs in a principled fashion. The core idea behind LexiCon is to take existing planning environments and impose temporal constraints on the states. These constrained problems are then translated into natural
language and given to an LLM to solve. A key feature of LexiCon is its extensibility. That is, the set of supported environments can be extended with new (unconstrained) environment generators, for which temporal constraints are constructed automatically. This renders LexiCon future-proof: the hardness of the generated planning problems can be increased as the planning capabilities of LLMs improve. Our experiments reveal that the performance of state-of-the-art LLMs, including
reasoning models like GPT-5, o3, and R1, deteriorates as the degree of constrainedness of the planning tasks increases.
Read the full abstract ▾
All authors Periklis Mantenoglou, Rishi Hazra, Pedro Zuidberg Dos Martires, Luc De Raedt
Computation and Language
Probing the Difficulty Perception Mechanism of Large Language Models
Sunbowen Lee, Qingyu Yin, Chak Tou Leong et al. (+5 more)
Large
language models (LLMs) are increasingly deployed on complex
reasoning tasks, yet little is known about their ability to internally evaluate problem difficulty, which is an essential capability for adaptive
reasoning and efficient resource allocation. In this work, we investigate whether LLMs implicitly encode problem difficulty in their internal representations. Using a linear probe on the final-
token representations of LLMs, we demonstrate that the difficulty level of math problems can be linearly modeled. We further locate the specific
attention heads of the final
Transformer layer: these
attention heads have opposite activation patterns for simple and difficult problems, thus achieving perception of difficulty. Our ablation experiments prove the accuracy of the location. Crucially, our experiments provide practical support for using LLMs as automatic difficulty annotators, potentially substantially reducing reliance on costly human labeling in
benchmark construction and curriculum learning. We also uncover that there is a significant difference in
entropy and difficulty perception at the
token level. Our study reveals that difficulty perception in LLMs is not only present but also structurally organized, offering new theoretical insights and practical directions for future research. Our code is available at https://github.com/Aegis1863/Difficulty-Perception-of-LLMs.
Read the full abstract ▾
All authors Sunbowen Lee, Qingyu Yin, Chak Tou Leong, Jialiang Zhang, Yicheng Gong, Shiwen Ni, Min Yang, Xiaoyu Shen
Computation and Language · Cross-listed from cs.AI
MatheMagic: Generating Dynamic Mathematics Benchmarks Robust to Memorization
Dayyán O'Brien, Barry Haddow, Emily Allaway et al. (+1 more)
Conducting contamination-free evaluation of mathematical capabilities can be difficult for two reasons: models may memorize a test set once it is made public, and current mathematical
benchmarks are prone to overfitting due to having limited diversity of symbols and rules, coupled with closed-ended answers. This paper proposes a method to leverage these shortcomings as useful features to a construct dynamic, counterfactual
benchmark, which can be used to both reveal overfitting and measure true
reasoning. We demonstrate this via MatheMagic, which generates math test instances with the interpretations of numbers and operators altered, yet has automatically verifiable answers. Test instances are randomly seeded and constructed at test time to evaluate a model's induction or deduction capability, offering stability, extensibility, comparability, and robustness to overfitting. Our experiments find that models solve deduction more easily than induction, but they revert to standard math. Further analysis reveals that math-adapted models fail to exhibit a general "skill" of
reasoning, and fine-tuning on induction tasks generalizes poorly.
Read the full abstract ▾
All authors Dayyán O'Brien, Barry Haddow, Emily Allaway, Pinzhen Chen
Computation and Language
EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models
Hadi Mohammadi, Anastasia Giachanou and Robert A. Bagheri
We present EvalMORAAL, a transparent chain-of-thought (CoT) framework that uses two scoring methods (log-probabilities and direct ratings) plus a model-as-judge peer review to evaluate moral
alignment in 20 large
language models. We assess models on the World Values Survey (55 countries, 19 topics) and the PEW Global Attitudes Survey (39 countries, 8 topics). With EvalMORAAL, top models align closely with survey responses (Pearson's $r \approx 0.90$ on WVS). Yet we find a clear regional difference: Western regions average $r=0.82$ while non-Western regions average $r=0.61$ (a 0.21 absolute gap), indicating a persistent regional
alignment gap. Our framework adds three parts: (1) two scoring methods for all models to enable fair comparison, (2) a structured CoT protocol with self-consistency checks, and (3) a model-as-judge peer review that flags 348 conflicts using a data-driven threshold. Peer agreement relates to WVS survey
alignment ($r=0.74$, $p<.001$; PEW $r=0.39$, n.s.), supporting automated quality checks. These results show real progress toward culture-aware AI while highlighting open challenges for use across regions.
Read the full abstract ▾
Computation and Language
Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lens
Mai AlKhamissi, Yunze Xiao, Badr AlKhamissi et al. (+1 more)
Cultural evaluation of large
language models has become increasingly important, yet current
benchmarks often reduce culture to static facts or homogeneous values. This view conflicts with anthropological accounts that emphasize culture as dynamic, historically situated, and enacted in practice. To analyze this gap, we introduce a four-part framework that categorizes how
benchmarks frame culture, such as knowledge, preference, performance, or bias. Using this lens, we qualitatively examine 20 cultural
benchmarks and identify six recurring methodological issues, including treating countries as cultures, overlooking within-culture diversity, and relying on oversimplified survey formats. Drawing on established anthropological methods, we propose concrete improvements: incorporating real-world narratives and scenarios, involving cultural communities in design and validation, and evaluating models in context rather than isolation. Our aim is to guide the development of cultural
benchmarks that go beyond static recall tasks and more accurately capture the responses of the models to complex cultural situations.
Read the full abstract ▾
All authors Mai AlKhamissi, Yunze Xiao, Badr AlKhamissi, Mona Diab
Computation and Language
Prompt reinforcing for long-term planning of large language models
Hsien-Chin Lin, Benjamin Matthias Ruppik, Carel van Niekerk et al. (+6 more)
Large
language models (LLMs) have achieved remarkable success in a wide range of natural
language processing tasks and can be adapted through prompting. However, they remain suboptimal in multi-turn interactions, often relying on incorrect early assumptions and failing to track user goals over time, which makes such tasks particularly challenging. Prior works in dialogue systems have shown that long-term planning is essential for handling interactive tasks. In this work, we propose a
prompt optimisation framework inspired by reinforcement learning, which enables such planning to take place by only modifying the task instruction
prompt of the LLM-based
agent. By generating turn-by-turn feedback and leveraging experience replay for
prompt rewriting, our proposed method shows significant improvement in multi-turn tasks such as text-to-SQL and task-oriented dialogue. Moreover, it generalises across different LLM-based
agents and can leverage diverse LLMs as meta-prompting
agents. This warrants future research in reinforcement learning-inspired parameter-free optimisation methods.
Read the full abstract ▾
All authors Hsien-Chin Lin, Benjamin Matthias Ruppik, Carel van Niekerk, Chia-Hao Shen, Michael Heck, Nurul Lubis, Renato Vukovic, Shutong Feng, Milica Gašić
Computation and Language · Cross-listed from cs.LG
Untangling Component Imbalance in Hybrid Linear Attention Conversion Methods
Martin Benfeghoul, Teresa Delgado, Adnan Oomerjee et al. (+3 more)
Transformers' quadratic computational complexity limits their scalability despite remarkable performance. While linear
attention reduces this to linear complexity, pre-training such models from scratch remains, in most cases, prohibitively expensive. Recent post-training linearisation methods convert pre-trained
Transformers to linear models efficiently, often using hybrid approaches that combine linear
attention with sliding-window softmax. We identify a critical flaw: existing hybrid methods inadvertently bypass the linear component, relying almost entirely on SWA. Component-level diagnostics reveal this previously undetected behaviour stems from overlooked evaluation practices on common-sense
benchmarks. We propose three solutions to ensure balanced component usage: (i) inference-time hybridisation of linear-only conversions with sliding-window softmax; (ii) HedgeCATs, combining
attention-weight transfer with targeted LoRA fine-tuning; and (iii) Scheduled Sliding-window Dropout (SSD), which stochastically suppresses the softmax branch during training to prevent component collapse. Our methods maintain computational efficiency while recovering most base model performance and ensuring genuine linear
attention adoption, restoring the validity of performance attributions in hybrid conversions.
Read the full abstract ▾
All authors Martin Benfeghoul, Teresa Delgado, Adnan Oomerjee, Haitham Bou Ammar, Jun Wang, Zafeirios Fountas
Computation and Language
The fragility of "cultural tendencies" in LLMs
Kun Sun and Rong Wang
In a recent study, Lu, Song, and Zhang (2025) (LSZ) propose that large
language models (LLMs), when prompted in different
languages, display culturally specific tendencies. They report that the two models (i.e., GPT and ERNIE) respond in more interdependent and holistic ways when prompted in Chinese, and more independent and analytic ways when prompted in English. LSZ attribute these differences to deep-seated cultural patterns in the models, claiming that
prompt language alone can induce substantial cultural shifts. While we acknowledge the empirical patterns they observed, we find their experiments, methods, and interpretations problematic. In this paper, we critically re-evaluate the methodology, theoretical framing, and conclusions of LSZ. We argue that the reported "cultural tendencies" are not stable traits but fragile artifacts of specific models and task design. To test this, we conducted targeted replications using a broader set of LLMs and a larger number of test items. Our results show that
prompt language has minimal effect on outputs, challenging LSZ's claim that these models encode grounded cultural beliefs.
Read the full abstract ▾
Computation and Language
On the Sensitivity of Instruction-tuned LLMs to Harmful Sentences in Long Inputs
Faeze Ghorbanpour and Alexander Fraser
Large
language models (LLMs) increasingly operate on long inputs, yet their behavior when harmful sentences are sparsely embedded within such inputs remains poorly understood. We present a sensitivity analysis that probes how LLMs extract harmful sentences embedded in long inputs. We construct long inputs by combining neutral and harmful sentences, and systematically vary four factors: input length (600–30,000
tokens), the proportion of harmful sentences (0.01–0.50), harm realization (explicit vs. implicit), and the position of harmful sentences within the input (beginning, middle, end), enabling a controlled stress-test evaluation. Experiments across toxic, offensive, and hate content, and across LLaMA-3.1, Qwen-2.5, and Mistral, reveal consistent patterns: sensitivity is non-monotonic with respect to harmful prevalence, peaking at moderate levels; sensitivity degrades as input length increases; harmful sentences placed earlier in the input are more strongly prioritized; and explicit harm is more reliably identified than implicit harm. These findings provide a systematic view of how LLMs prioritize harmful sentences in long input under controlled stress conditions, highlighting both emerging strengths and remaining challenges for safety-related use.
Read the full abstract ▾
Computation and Language
Revisiting Long-context Modeling from Context Denoising Perspective
Zecheng Tang, Baibei Ji, Juntao Li et al. (+3 more)
Long-context models (LCMs) have demonstrated great potential in processing long sequences, facilitating many real-world applications. The success of LCMs can be attributed to their ability to locate implicit critical information within the context for further prediction. However, recent research reveals that LCMs are often susceptible to contextual noise, i.e., irrelevant
tokens, that can mislead model
attention. In this paper, we conduct a fine-grained analysis of the context noise and propose an effective metric, the Integrated Gradient (IG) score, to detect and quantify the noise information within the context. Our findings reveal that even simple mitigation of detected context noise can substantially boost the model's
attention on critical
tokens and benefit subsequent predictions. Building on this insight, we propose Context Denoising Training (CDT), a straightforward yet effective training strategy that improves
attention on critical
tokens while reinforcing their influence on model predictions. Extensive experiments across four tasks, under both context window scaling and long-context
alignment settings, demonstrate the superiority of CDT. Notably, when trained with CDT, an open-source 8B model can achieve performance (50.92) comparable to GPT-4o (51.00).
Read the full abstract ▾
All authors Zecheng Tang, Baibei Ji, Juntao Li, Lijun Wu, Haijia Gui, Min Zhang
Computation and Language
Overcoming Language Barriers: Multilingual Analysis of the 2023 Swiss Privacy Law's Impact
Luka Nenadic, David Rodriguez and Joseph A. Calandrino
Policymakers enact and revise
privacy laws expecting meaningful benefits for their people in practice. While scholarship has measured the real-world impact of some
privacy regulations-the EU and California most notably-limited empirical evidence exists for many of the more than 140 countries that have implemented some form of
privacy legislation. Switzerland, a
multilingual country bordered almost entirely by EU states, is one such example. This paper analyzes the extent to which a 2023
alignment of Swiss
privacy law with EU
privacy regulation affected website
privacy policies in Switzerland. To address Switzerland's unique
multilingual culture, we develop an LLM-based pipeline that extracts legally relevant information as document-level labels in a single inference without requiring translation. On a
benchmark of 120 expert-annotated
privacy policies in German, French, Italian, and English, our pipeline achieves F1 scores above 0.90 for most pairs of
languages and legally relevant disclosures. Applying this pipeline to
privacy policies we collected from more than 35,000 Swiss- and EU-facing websites before and after the 2023
privacy law revision, we find significant increases in both mandatory and voluntary disclosures of data subject rights among Swiss
privacy policies. In exploring the mechanisms driving increased disclosure rates, we discover heavy use of automated
privacy policy generators and find that generated policies are associated with up to 15 percentage points higher disclosure rates. These results provide large-scale empirical evidence of how regulatory change and novel drafting technologies impact the content of
privacy policies in a unique
multilingual environment.
Read the full abstract ▾
Computation and Language
DACP: Domain-Adaptive Continual Pre-Training of Large Language Models for Phone Conversation Summarization
Xue-Yong Fu, Elena Khasanova, Md Tahmid Rahman Laskar et al. (+2 more)
Large
language models (LLMs) have achieved impressive performance in text summarization, yet their performance often falls short when applied to specialized domains that differ from their original pre-training distribution. While fine-tuning can improve summarization quality, it typically relies on costly and scarce high-quality labeled data. In this work, we explore continual pre-training as a scalable, self-supervised approach to adapt LLMs for downstream summarization tasks, particularly in the context of noisy real-world conversation transcripts. We conduct extensive experiments using large-scale, unlabeled business conversation data to investigate whether continual pre-training enhances model capabilities in conversational summarization. Our results demonstrate that continual pre-training yields substantial gains in both in-domain and out-of-domain summarization
benchmarks, while maintaining strong generalization and robustness. We also analyze the effects of data selection strategies, providing practical guidelines for applying continual pre-training in summarization-focused industrial applications.
Read the full abstract ▾
All authors Xue-Yong Fu, Elena Khasanova, Md Tahmid Rahman Laskar, Harsh Saini, Shashi Bhushan TN
Computation and Language
Luth: Efficient French Specialization for Small Language Models and Cross-Lingual Transfer
Maxence Lasbordes and Sinoué Gad
The landscape of Large
Language Models remains predominantly English-centric, resulting in a significant performance gap for other major
languages, such as French, especially in the context of Small
Language Models (SLMs). Existing
multilingual models demonstrate considerably lower performance in French compared to English, and research on efficient adaptation methods for French remains limited. To address this, we introduce Luth, a family of French-specialized SLMs: through targeted post-training on curated, high-quality French data, our models outperform all open-source counterparts of comparable size on multiple French
benchmarks while retaining their original English capabilities. We further show that strategic model merging enhances performance in both
languages, establishing Luth as a new state of the art for French SLMs and a robust baseline for future French-
language research.
Read the full abstract ▾
Computation and Language
EEPO: Exploration-Enhanced Policy Optimization via Sample-Then-Forget
Liang Chen, Xueting Han, Qizhou Wang et al. (+4 more)
Balancing exploration and exploitation remains a central challenge in reinforcement learning with verifiable rewards (RLVR) for large
language models (LLMs). Current RLVR methods often overemphasize exploitation, leading to
entropy collapse, diminished exploratory capacity, and ultimately limited performance gains. Although techniques that increase policy stochasticity can promote exploration, they frequently fail to escape dominant behavioral modes. This creates a self-reinforcing loop – repeatedly sampling and rewarding dominant modes – that further erodes exploration. We introduce Exploration-Enhanced Policy Optimization (EEPO), a framework that promotes exploration via two-stage rollouts with adaptive unlearning. In the first stage, the model generates half of the trajectories; it then undergoes a lightweight unlearning step to temporarily suppress these sampled responses, forcing the second stage to explore different regions of the output space. This sample-then-forget mechanism disrupts the self-reinforcing loop and promotes wider exploration during rollouts. Across five
reasoning benchmarks, EEPO outperforms GRPO, achieving average relative gains of 24.3% on Qwen2.5-3B, 33.0% on Llama3.2-3B-Instruct, and 10.4% on Qwen3-8B-Base.
Read the full abstract ▾
All authors Liang Chen, Xueting Han, Qizhou Wang, Bo Han, Jing Bai, Hinrich Schutze, Kam-Fai Wong
Computation and Language · Cross-listed from cs.LG
Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scaling
Giorgio Giannone, Guangxuan Xu, Nikhil Shivakumar Nayak et al. (+4 more)
Inference-Time Scaling (ITS) improves
language models by allocating more computation at generation time. Particle Filtering (PF) has emerged as a strong ITS method for complex mathematical
reasoning tasks, but it is vulnerable when guided by process reward models, which often assign overconfident scores early in the
reasoning process. This causes PF to suffer from premature exploitation: it myopically commits to locally promising trajectories, prunes potentially correct hypotheses, and converges to suboptimal solutions. This failure mode, known as particle impoverishment, is especially severe under constrained computational budgets. To address this, we analyze the problem and identify two root causes: a lack of diversity in the particle set due to overconfident resampling and consequent inability to assess the potential of a
reasoning path. We introduce Entropic Particle Filtering (ePF), an
algorithm that integrates two new techniques to solve these issues. The first technique, Entropic Annealing (EA), directly mitigates particle impoverishment by monitoring search diversity via
entropy; when diversity drops, it intervenes by dynamically annealing the resampling distribution to preserve exploration. The second, an enhancement called Look-ahead Modulation (LaM), adds a predictive guide to evaluate a state's potential based on its successors. On several challenging math
benchmarks, ePF significantly outperforms strong baselines and achieves up to a 50% relative improvement in task reward. Together, these methods improve PF's resilience by balancing the exploration of diverse solution spaces with the exploitation of high-reward regions, ultimately leading to higher-quality solutions.
Read the full abstract ▾
All authors Giorgio Giannone, Guangxuan Xu, Nikhil Shivakumar Nayak, Rohan Mahesh Awhad, Shivchander Sudalairaj, Kai Xu, Akash Srivastava