Computation and Language
Directed Social Regard: Surfacing Targeted Advocacy, Opposition, Aid, Harms, and Victimization in Online Media
Scott Friedman, Ruta Wheelock, Sonja Schmer-Galunder et al. (+6 more)
The
language in online platforms, influence operations, and political rhetoric frequently directs a mix of pro-social sentiment (e.g., advocacy, helpfulness, compassion) and anti-social sentiment (e.g., threats, opposition, blame) at different topics, all in the same message. While many natural
language processing (NLP) tools classify or score a text's overall sentiment as positive, neutral, or negative, these tools cannot report that positive and negative sentiments coexist, and they cannot report the target of those sentiments. This paper presents the Directed Social Regard (DSR) approach to multi-dimensional, multi-valence sentiment analysis, comprised of a pair of
transformer-based models that (1) detects span-level targets of sentiment in a message and then (2) scores all spans within the message context along three (-1, 1) axes of regard that are motivated by social science theories of moral disengagement and moral framing. We present a data collection and annotation strategy for DSR
dataset construction, a
transformer-based architecture for span-level scoring, and a validation study with promising results. We apply the validated DSR model on six third-party
datasets of online media and report meaningful correlations between DSR outputs and the labels and topics in these pre-existing social science
datasets.
Read the full abstract ▾
All authors Scott Friedman, Ruta Wheelock, Sonja Schmer-Galunder, Drisana Iverson, Jake Vasilakes, Joan Zheng, Jeffrey Rye, Vasanth Sarathy, Christopher Miller
Computation and Language
Characterizing the Expressivity of Local Attention in Transformers
Jiaoda Li and Ryan Cotterell
The
transformer is the most popular neural architecture for
language modeling. The cornerstone of the
transformer is its global
attention mechanism, which lets the model aggregate information from all preceding
tokens before generating the next
token. One common variant of
attention is called local
attention, which restricts each
token to aggregating information from a bounded window of predecessors, reducing the quadratic cost of global
attention to linear. Although this restriction is usually motivated by efficiency, it has also been found to improve model quality, a phenomenon that has so far lacked a satisfactory explanation. We provide a formal account of this phenomenon in terms of recognizer expressivity. It has been shown that fixed-precision
transformers with global
attention correspond to a fragment of linear temporal logic containing a single past operator. We additionally prove that adding local
attention introduces a second temporal operator, strictly enlarging the class of recognizable regular
languages. Moreover, global and local
attention are expressively complementary: neither subsumes the other, and combining them yields the richest fragment. Experiments on formal
language recognition and natural
language modeling corroborate the theory, showing that hybrid global–local
transformers outperform their global-only counterparts.
Read the full abstract ▾
Computation and Language
FinSafetyBench: Evaluating LLM Safety in Real-World Financial Scenarios
Yutao Hou, Yihan Jiang, Yuhan Xie et al. (+5 more)
Large
language models (LLMs) are increasingly applied in financial scenarios. However, they may produce harmful outputs, including facilitating illegal activities or unethical behavior, posing serious compliance risks. To systematically evaluate LLM safety in finance, we propose FinSafetyBench, a bilingual (English-Chinese) red-teaming
benchmark designed to test an LLM's refusal of requests that violate financial compliance. Grounded in real-world financial crime cases and ethics standards, the
benchmark comprises 14 subcategories spanning financial crimes and ethical violations. Through extensive experiments on general-purpose and finance-specialized LLMs under three representative attack settings, we identify critical vulnerabilities that allow
adversarial prompts to bypass compliance safeguards. Further analysis reveals stronger susceptibility in Chinese contexts and highlights the limitations of
prompt-level defenses against sophisticated or implicit manipulation strategies.
Read the full abstract ▾
All authors Yutao Hou, Yihan Jiang, Yuhan Xie, Jian Yang, Liwen Zhang, Hailiang Huang, Guanhua Chen, Yun Chen
Computation and Language
Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memory
Derong Xu, Shuochen Liu, Pengfei Luo et al. (+8 more)
Large
language model (LLM)
agents require long-term user
memory for consistent personalization, but limited context windows hinder tracking evolving preferences over long interactions. Existing
memory systems mainly rely on static, hand-crafted update rules; although reinforcement learning (RL)-based
agents learn
memory updates, sparse outcome rewards provide weak supervision, resulting in unstable long-horizon optimization. Drawing on
memory schema theory and the functional division between prefrontal regions and hippocampus regions, we introduce MemCoE, a cognition-inspired two-stage optimization framework that learns how
memory should be organized and what information to update. In the first stage, we propose
Memory Guideline Induction to optimize a global guideline via contrastive feedback interpreted as textual gradients; in the second stage, Guideline-Aligned
Memory Policy Optimization uses the induced guideline to define structured process rewards and performs multi-turn RL to learn a guideline-following
memory evolution policy. We evaluate on three personalization
memory benchmarks, covering explicit/implicit preference and different sizes and noise, and observe consistent improvements over strong baselines with favorable robustness, transferability, and efficiency.
Read the full abstract ▾
All authors Derong Xu, Shuochen Liu, Pengfei Luo, Pengyue Jia, Yingyi Zhang, Yi Wen, Yimin Deng, Wenlin Zhang, Enhong Chen, Xiangyu Zhao, Tong Xu
Computation and Language · Cross-listed from stat.ML
Adaptive Querying with AI Persona Priors
Kaizheng Wang, Yuhang Wu and Assaf Zeevi
We study adaptive querying for learning user-dependent quantities of interest, such as responses to held-out items and psychometric indicators, within tight query budgets. Classical
Bayesian design and computerized adaptive testing typically rely on restrictive parametric assumptions or expensive posterior approximations, limiting their use in heterogeneous, high-dimensional, and cold-start settings. We introduce a persona-induced latent variable model that represents a user's state through membership in a finite dictionary of AI personas, each offering response distributions produced by a large
language model. This yields expressive priors with closed-form posterior updates and efficient finite-mixture predictions, enabling scalable
Bayesian design for sequential item selection. Experiments on synthetic data and WorldValuesBench demonstrate that persona-based posteriors deliver accurate probabilistic predictions and an interpretable adaptive elicitation pipeline.
Read the full abstract ▾
Computation and Language
BabelSafe: A Policy-Grounded Multilingual Safety Benchmark for LLMs
Yunhan Zhao, Zhaorun Chen, Xingjun Ma et al. (+1 more)
As Large
Language Models (LLMs) are increasingly deployed in cross-linguistic contexts, ensuring safety across diverse regulatory and cultural environments has become a critical challenge. However, existing
multilingual benchmarks largely rely on general risk taxonomies and machine-translated data, limiting evaluation to predefined risk categories and providing insufficient coverage of region-specific regulatory requirements and cultural contexts. To bridge these gaps, we introduce BabelSafe, a policy-grounded
multilingual safety
benchmark covering 13
language settings. BabelSafe is constructed from regional regulatory sources, with risk categories and fine-grained rules extracted from jurisdiction-specific regulatory documents directly used to guide the generation of
multilingual safety data. During data generation, we further incorporate region-specific cultural contexts, enabling regulation-grounded and culturally contextualized evaluation across
languages. Building on BabelSafe, we develop BabelGuard, a
Diffusion Large
Language Model (dLLM)-based guardrail model that supports
multilingual safety judgment and policy-conditioned safety assessment. BabelGuard has two variants, a lightweight 1.5B model for fast `safe/unsafe' classification and a more capable 7B model for customizable policy-conditioned safety checking with detailed explanations. We evaluate BabelGuard against 11 strong guardrail baselines on 6 existing
multilingual safety
benchmarks and BabelSafe, demonstrating the strong performance of BabelGuard across these evaluation settings. We hope that BabelSafe and BabelGuard can help advance the development of regulation-aware and culturally contextualized
multilingual guardrail systems.
Read the full abstract ▾
All authors Yunhan Zhao, Zhaorun Chen, Xingjun Ma, Bo Li
Computation and Language
Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs
Jasper Dekoninck, Nikola Jovanović, Tim Gehrunger et al. (+4 more)
Large
language models (LLMs) are becoming increasingly capable mathematical collaborators, but static
benchmarks are no longer sufficient for evaluating progress: they are often narrow in scope, quickly saturated, and rarely updated. This makes it hard to compare models reliably and track progress over time. Instead, we need evaluation platforms: continuously maintained systems that run, aggregate, and analyze evaluations across many
benchmarks to give a comprehensive picture of model performance within a broad domain. In this work, we build on the original MathArena
benchmark by substantially broadening its scope from final-answer olympiad problems to a continuously maintained evaluation platform for mathematical
reasoning with LLMs. MathArena now covers a much wider range of tasks, including proof-based competitions, research-level arXiv problems, and formal proof generation in Lean. Additionally, we maintain a clear evaluation protocol for all models and regularly design new
benchmarks as model capabilities improve to ensure that MathArena remains challenging. Notably, the strongest model, GPT-5.5, now reaches 98% on the 2026 USA Math Olympiad and 74% on research-level questions, showing that frontier models can now comfortably solve extremely challenging mathematical problems. This highlights the importance of continuously maintained evaluation platforms like MathArena to track the rapid progress of LLMs in mathematical
reasoning.
Read the full abstract ▾
All authors Jasper Dekoninck, Nikola Jovanović, Tim Gehrunger, Kári Rögnvaldsson, Ivo Petrov, Chenhao Sun, Martin Vechev
Computation and Language
H-RAG at SemEval-2026 Task 8: Hierarchical Parent-Child Retrieval for Multi-Turn RAG Conversations
Passant Elchafei, Hossam Emam, Mohamed Alansary et al. (+2 more)
We present H-RAG, our submission to SemEval-2026 Task 8 (MTRAGEval), addressing both Task A (
Retrieval) and Task C (Generation with Retrieved Passages). Task A evaluates standalone
retrieval quality, while Task C assesses end-to-end
retrieval-augmented generation (RAG) in multi-turn conversational settings, requiring both accurate answer generation and faithful grounding in retrieved evidence. Our approach implements a hierarchical parent-child RAG pipeline that separates fine-grained child-level
retrieval from parent-level context reconstruction during generation. Documents are segmented into overlapping sentence-based child chunks, while full documents are preserved as parent units to provide coherent context.
Retrieval combines hybrid dense-sparse search, tunable weighting, and
embedding-based similarity rescoring over child chunks. Retrieved evidence is aggregated at the parent level and supplied to an instruction-tuned
language model for response generation. H-RAG achieves an nDCG@5 score of 0.4271 on Task A and a harmonic mean score of 0.3241 on Task C (RB_agg: 0.2488, RL_F: 0.2703, RB_llm: 0.6508), underscoring the importance of
retrieval configuration and parent-level aggregation in multi-turn RAG performance.
Read the full abstract ▾
All authors Passant Elchafei, Hossam Emam, Mohamed Alansary, Monorama Swain, Markus Schedl
Computation and Language · Cross-listed from cs.DB
EGREFINE: An Execution-Grounded Optimization Framework for Text-to-SQL Schema Refinement
Jiaqian Wang, Yutao Qi, Wenjin Hou et al. (+2 more)
Text-to-SQL enables non-expert users to query databases in natural
language, yet real-world schemas often suffer from ambiguous, abbreviated, or inconsistent naming conventions that degrade model accuracy. Existing approaches treat schemas as fixed and address errors downstream. In this paper, we frame schema refinement as a constrained optimization problem: find a renaming function that maximizes downstream Text-to-SQL execution accuracy while preserving query equivalence through database views. We analyze the computational hardness of this problem, which motivates a column-wise greedy decomposition, and instantiate it as EGRefine: a four-phase pipeline that screens ambiguous columns, generates context-aware candidate names, verifies them through execution-grounded feedback, and materializes the result as non-destructive SQL views. The pipeline carries two structural properties: column-local non-degradation, ensured by the conservative selection rule in the verification phase, and database-level query equivalence, ensured by the view-based materialization phase. Together they make the resulting refinement safe by construction at the column level, with cross-column and
prompt-level interactions handled empirically rather than analytically. Across controlled schema-degradation, real-world, and enterprise
benchmarks, EGRefine recovers accuracy lost to schema naming noise where applicable and correctly abstains where the underlying task exceeds current Text-to-SQL capabilities, with refined schemas transferring across model families to enable refine-once, serve-many-models deployment. Code and data are publicly available at https://github.com/ai-jiaqian/EGRefine.
Read the full abstract ▾
All authors Jiaqian Wang, Yutao Qi, Wenjin Hou, Yu Pang, Rui Yang
Computation and Language
SC-Taxo: Hierarchical Taxonomy Generation under Semantic Consistency Constraints using Large Language Models
Shiqiang Cai, Nianhong Niu, Shizhu He et al. (+2 more)
Scientific literature is expanding at an unprecedented pace, making it increasingly challenging to efficiently organize and access domain knowledge. A high-quality scientific taxonomy offers a structured and hierarchical representation of a research field, facilitating literature exploration and topic
navigation, as well as enabling downstream applications such as trend analysis, idea generation, and information
retrieval. However, existing taxonomy generation approaches often suffer from structural inconsistencies and semantic misalignment across hierarchical levels. Through empirical analysis, we find that these issues largely stem from inadequate modeling of hierarchical semantic consistency. To address this limitation, we propose a semantic-consistent taxonomy generation (SC-Taxo) framework that leverages large
language models (LLMs) with hierarchy-aware refinement stages to ensure semantic consistency. Specifically, SC-Taxo introduces a bidirectional heading generation mechanism that jointly performs bottom-up abstraction and top-down semantic constraint, while further capturing peer-level semantic dependencies to enhance horizontal consistency. Experiments on multiple
benchmark datasets demonstrate consistent improvements in hierarchy
alignment and heading quality, and additional evaluation on Chinese scientific literature validates its robust cross-lingual generalization.
Read the full abstract ▾
All authors Shiqiang Cai, Nianhong Niu, Shizhu He, Kang Liu, Jun Zhao
Computation and Language
Is Textual Similarity Invariant under Machine Translation? Evidence Based on the Political Manifesto Corpus
Daria Boratyn, Damian Brzyski, Albert Leśniak et al. (+5 more)
We investigate the extent to which cosine similarity between paragraph
embeddings is invariant under machine translation, using the Manifesto Corpus of over 2,800 political party platforms in 28
languages translated to English via the EU eTranslation service. Rather than measuring translation-induced semantic shift directly we measure the stability of pairwise similarity relationships across
embedding models, and use inter-model disagreement on original-
language text as a calibrated invariance threshold. This yields a per-
language non-inferiority test for four hypotheses about how translation interacts with
embedding choice, with verdicts that distinguish
languages where translation demonstrably preserves semantic structure from those where it demonstrably degrades it and from those where the available evidence does not resolve the question. The framework is corpus- and pipeline-agnostic and extends naturally to downstream tasks. Applied to our data, it identifies ten
languages with translation invariance and four with detectable distortion.
Read the full abstract ▾
All authors Daria Boratyn, Damian Brzyski, Albert Leśniak, Wojciech Łukasik, Maciej Rapacz, Jan Rybicki, Wojciech Słomczyński, Dariusz Stolicki
Computation and Language
Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe
Gaofei Shen, Martijn Bentum, Tomas O. Lentz et al. (+2 more)
Probing is widely used to study which features can be decoded from
language model representations. However, the common decoding probe approach has two limitations that we aim to solve with our new encoding probe approach: contributions of different features to model representations cannot be directly compared, and feature correlations can affect probing results. We present an Encoding Probe that reverses this direction and reconstructs internal representations of models using interpretable features. We evaluate this method on text and
speech transformer models, using feature sets spanning acoustics, phonetics, syntax, lexicon, and speaker identity. Our results suggest that speaker-related effects vary strongly across different training objectives and
datasets, while syntactic and lexical features contribute independently to reconstruction. These results show that the Encoding Probe provides a complementary perspective on interpreting model representations beyond decodability.
Read the full abstract ▾
All authors Gaofei Shen, Martijn Bentum, Tomas O. Lentz, Afra Alishahi, Grzegorz Chrupała
Computation and Language
Structure Liberates: How Constrained Sensemaking Produces More Novel Research Output
James Mooney, Zae Myung Kim, Young-Jun Lee et al. (+1 more)
Scientific discovery is an extended process of ideation – surveying prior work, forming hypotheses, and refining
reasoning – yet existing LLM research
agents treat this phase as a brief preamble despite its central role in research. We introduce SciSense, a sensemaking-grounded framework that operationalizes ideation as a structured sequence of eight cognitive stages (Pirolli and Card, 2005). We construct SciSense-Traj, a
dataset of 120K citation-conditioned research trajectories in two modes: Hindsight, where an LLM reconstructs the ideation path from a paper's cited works to the paper itself, and Foresight, where the LLM proposes novel directions from the same citations. We distill these into SciSense-LM, a family of sensemaking LLMs spanning 3B to 70B parameters. Contrary to the assumption that looser supervision promotes exploration, Hindsight-trained models improve trajectory quality by 2.0% over Foresight-trained models while also producing more novel and diverse outputs, across all eight model families we test. The advantage propagates downstream: coding
agents conditioned on Hindsight trajectories produce research artifacts with higher executability, scientific grounding, and utility than those conditioned on Foresight trajectories. This suggests that outcome-anchored ideation reduces the cognitive burden on downstream
agents, freeing them to explore more creatively. SciSense offers both a practical tool for augmenting LLM-driven research workflows and a principled testbed for studying how planning shapes scientific discovery.
Read the full abstract ▾
All authors James Mooney, Zae Myung Kim, Young-Jun Lee, Dongyeop Kang
Computation and Language
A11y-Compressor: A Framework for Enhancing the Efficiency of GUI Agent Observations through Visual Context Reconstruction and Redundancy Reduction
Michito Takeshita, Takuro Kawada, Takumi Ohashi et al. (+2 more)
AI
agents that interact with graphical user interfaces (GUIs) require effective observation representations for reliable grounding. The accessibility tree is a commonly used text-based format that encodes UI element attributes, but it suffers from redundancy and lacks structural information such as spatial relationships among elements. We propose A11y-Compressor, a framework that transforms linearized accessibility trees into compact and structured representations. Our implementation, Compressed-a11y, applies a lightweight and structured transformation pipeline with modal detection, redundancy reduction, and semantic structuring. Experiments on the OSWorld
benchmark show that Compressed-a11y reduces input
tokens to 22% of the original while improving task success rates by 5.1 percentage points on average.
Read the full abstract ▾
All authors Michito Takeshita, Takuro Kawada, Takumi Ohashi, Shunsuke Kitada, Hitoshi Iyatomi
Computation and Language
AGoQ: Activation and Gradient Quantization for Memory-Efficient Distributed Training of LLMs
Wenxiang Lin, Juntao Huang, Luhan Zhang et al. (+5 more)
Quantization is a key method for reducing the GPU
memory requirement of training large
language models (LLMs). Yet, current approaches are ineffective for 4-bit activations and 8-bit gradients, which would easily cause slow convergence or accuracy loss. To address this, we introduce AGoQ, incorporating two new techniques: 1) a layer-aware activation quantization
algorithm that allocates appropriate bit-widths for activations of various layers based on their types and pipeline stages to achieve near 4-bit activation storage, and 2) a gradient quantization
algorithm that reduces
memory usage and shortens communication time by employing 8-bit gradient storage and precision-preserving 8-bit All-Reduce communication. We conduct extensive experiments using different sizes of LLMs on two GPU clusters (up to 64 GPUs), and the experimental results show that our AGoQ reduces the
memory by up to 52% and achieves up to 1.34$\times$ improvement of training speed compared to state-of-the-art training systems Megatron-LM (w/ or w/o ZeRO), COAT and DeepSpeed with 8B to 32B LLaMA models, while achieving convergence loss on pretraining and comparable accuracy on downstream tasks with LLaMA architectures.
Read the full abstract ▾
All authors Wenxiang Lin, Juntao Huang, Luhan Zhang, Laili Li, Xiang Bao, Mengyang Zhang, Bing Wang, Shaohuai Shi
Computation and Language
ControBench: An Interaction-Aware Benchmark for Controversial Discourse Analysis on Social Networks
Ta Thanh Thuy, Jiaqi Zhu, Xuan Liu et al. (+6 more)
Understanding how people argue across ideological divides online is important for studying political polarization, misinformation, and content moderation. Existing
datasets capture only part of this problem: some preserve text but ignore interaction structure, some model structure without rich semantics, and others represent conversations without stable user-level ideological identity. We introduce ControBench, a
benchmark for controversial discourse analysis that combines heterogeneous social interaction graphs with rich textual semantics. Built from Reddit discussions on three topics, Trump, abortion, and religion, ControBench contains 7,370 users, 1,783 posts, and 26,525 interactions. The graph contains user and post nodes connected by semantically enriched edges; in particular, user-comment-user edges encode both a reply and the parent comment that it responds to, preserving local argumentative context. User labels are derived from self-declared Reddit flairs, providing a scalable proxy for ideological identity without manual annotation. The resulting
datasets exhibit low or negative adjusted homophily (Trump: -0.77, Abortion: 0.06, Religion: 0.04), reflecting the cross-cutting structure of real-world debate. We evaluate graph neural networks, pretrained
language models, and large
language models on ControBench and observe distinct performance patterns across topics and model families, especially when ideological boundaries are ambiguous. These results position ControBench as a challenging and realistic
benchmark for controversial discourse analysis.
Read the full abstract ▾
All authors Ta Thanh Thuy, Jiaqi Zhu, Xuan Liu, Lin Shang, Reihaneh Rabbany, Guillaume Rabusseau, Lihui Chen, Zheng Yilun, Sitao Luan
Computation and Language
Surprisal Minimisation over Goal-directed Alternatives Predicts Production Choice in Dialogue
Tom Utting, Mario Giulianelli and Arabella Sinclair
We model utterance production as probabilistic cost-sensitive choice over contextual alternatives, using information-theoretic notions of cost. We distinguish between goal-directed alternatives that realise a fixed communicative intent and goal-agnostic alternatives defined only by contextual plausibility, allowing us to derive speaker- and listener-oriented interpretations of different cost measures. We present a procedure to generate both types of alternative sets using
language models. Analysing production choices in open-ended dialogue under both deterministic and probabilistic cost minimisation, we find that surprisal minimisation relative to goal-directed alternatives provides the strongest predictive account under both analyses. By contrast, uniform information density and length-based costs exhibit weaker and less consistent predictive power across conditions. More broadly, our study suggests that alternative-conditioned optimisation with LM-generated alternatives provides a principled framework for studying speaker and listener pressures in naturalistic
language production.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
LLM-Oriented Information Retrieval: A Denoising-First Perspective
Lu Dai, Liang Sun, Fanpu Cao et al. (+4 more)
Modern information
retrieval (IR) is no longer consumed primarily by humans but increasingly by large
language models (LLMs) via
retrieval-augmented generation (RAG) and agentic search. Unlike human users, LLMs are constrained by limited
attention budgets and are uniquely vulnerable to noise; misleading or irrelevant information is no longer just a nuisance, but a direct cause of
hallucinations and
reasoning failures. In this perspective paper, we argue that denoising-maximizing usable evidence density and verifiability within a context window-is becoming the primary bottleneck across the full information access pipeline. We conceptualize this paradigm shift through a four-stage framework of IR challenges: from inaccessible to undiscoverable, to misaligned, and finally to unverifiable. Furthermore, we provide a pipeline-organized taxonomy of signal-to-noise optimization techniques, spanning indexing,
retrieval, context engineering, verification, and agentic workflow. We also present research works on information denoising in domains that rely heavily on
retrieval such as lifelong assistant, coding
agent, deep research, and
multimodal understanding.
Read the full abstract ▾
All authors Lu Dai, Liang Sun, Fanpu Cao, Ziyang Rao, Cehao Yang, Hao Liu, Hui Xiong
Computation and Language · Cross-listed from cs.HC
"What Are You Really Trying to Do?": Co-Creating Life Goals from Everyday Computer Use
Shardul Sapkota, Matthew Jörke, Zane Sabbagh et al. (+3 more)
Recent advances in user modeling make it feasible to conduct open-ended inference over a person's everyday computer use. Despite longstanding
visions of systems that deeply understand our actions and the purposes they serve in our lives, existing systems only capture what a person is doing in the moment, not why they are doing it, limiting these systems to surface-level support. We introduce striving co-creation, a process for inferring broader life goals from unstructured observations of computer use. Grounded in Activity Theory and Emmons' personal strivings framework, our system progressively constructs a hierarchical representation of a person's activities. Strivings are, however, difficult to fully resolve from observation alone, as the same action can be driven by many different goals. Our system therefore supports an editing interface that gives people agency over how they are understood by the system, feeding their corrections back into subsequent rounds of striving induction. In a week-long field deployment (N=14), we find that our co-creation process produces strivings that participants recognize as representative of their long-term goals and gives them greater agency than baseline methods.
Read the full abstract ▾
All authors Shardul Sapkota, Matthew Jörke, Zane Sabbagh, Omar Shaikh, Grace Wang, James A. Landay
Computation and Language
ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?
Joey Chan, Yikun Han, Jingyuan Chen et al. (+8 more)
Plain
Language Summaries (PLS) aim to make research accessible to lay readers, but they are typically written in a one-size-fits-all style that ignores differences in readers' information needs and comprehension. In health contexts, this limitation is particularly important because misunderstanding scientific information can affect real-world decisions. Large
language models (LLMs) offer new opportunities for personalizing PLS, but it remains unclear whether personalization helps, which strategies are most effective, and how to balance personalization with safety. We introduce ReLay, a
dataset of 300 participant–PLS pairs from 50 lay participants in both static (expert-written) and interactive (LLM-personalized) settings. ReLay includes user characteristics, health information needs, information-seeking behavior, comprehension outcomes, interaction logs, and quality ratings. We use ReLay to evaluate five LLMs across two personalization methods. Personalization improves comprehension and perceived quality, but it also raises the risk of reinforcing user biases and introducing
hallucinations, revealing a trade-off between personalization and safety. These findings highlight the need for personalization methods that are both effective and trustworthy for diverse lay audiences.
Read the full abstract ▾
All authors Joey Chan, Yikun Han, Jingyuan Chen, Samuel Fang, Lauren D. Gryboski, Alexandra Lee, Sheel Tanna, Qingqing Zhu, Zhiyong Lu, Lucy Lu Wang, Yue Guo
Computation and Language · Cross-listed from cs.AI
Role-Aware Artificial Intelligence Across Augmentation and Automation in Human-Machine Symbiosis
Ching-Chun Chang, Yuchen Guo, Hanrui Wang et al. (+2 more)
The
evolution of artificial intelligence (AI) has rendered the boundary between humanity and computational machinery increasingly ambiguous. In the presence of more interwoven relationships within human-machine symbiosis, the very notion of AI-generated information becomes difficult to define, as such information arises not from either humans or machines in isolation, but from their mutual shaping. At times AI acts in place of the human, automating the task; at others it extends what the human can do, augmenting their capability. Therefore, a more pertinent question lies not merely in whether AI has participated, but in how it has participated. In general, the role assumed by AI is often specified, either implicitly or explicitly, in the input
prompt, yet becomes less apparent or altogether unobservable when the generated content alone is available. Once detached from the dialogue context, the functional role may no longer be traceable. This study considers the problem of tracing the functional role played by AI in natural
language generation. A methodology is proposed to infer the latent role specified by the
prompt, embed this role into the content during the probabilistic generation process and subsequently recover the nature of AI participation from the resulting text. Experimentation is conducted under a representative scenario in which AI acts either as an assistive
agent that edits human-written content or as a creative
agent that generates new content from a brief concept. The experimental results support the validity of the proposed methodology in terms of discrimination between roles, robustness against perturbations and preservation of linguistic quality. We envision that this study may contribute to future research on the ethics of AI with regard to whether AI has been used fairly, transparently and appropriately.
Read the full abstract ▾
All authors Ching-Chun Chang, Yuchen Guo, Hanrui Wang, Timo Spinde, Isao Echizen
Computation and Language
Impact of Task Phrasing on Presumptions in Large Language Models
Kenneth J.K. Ong
Concerns with the safety and reliability of applying large-
language models (LLMs) in unpredictable real-world applications motivate this study, which examines how task phrasing can lead to presumptions in LLMs, making it difficult for them to adapt when the task deviates from these assumptions. We investigated the impact of these presumptions on the performance of LLMs using the iterated prisoner's dilemma as a case study. Our experiments reveal that LLMs are susceptible to presumptions when making decisions even with
reasoning steps. However, when the task phrasing was neutral, the models demonstrated logical
reasoning without much presumptions. These findings highlight the importance of proper task phrasing to reduce the risk of presumptions in LLMs.
Read the full abstract ▾
Computation and Language
Escaping Mode Collapse in LLM Generation via Geometric Regulation
Xin Du and Kumiko Tanaka-Ishii
Mode collapse is a persistent challenge in generative modeling and appears in autoregressive text generation as behaviors ranging from explicit looping to gradual loss of diversity and premature trajectory convergence. We take a dynamical-systems view and reinterpret mode collapse as reduced state-space accessibility caused by *
geometric collapse*: during generation, the model's internal trajectory becomes confined to a low-dimensional region of its representation space. This implies mode collapse is not purely a
token-level phenomenon and cannot be reliably solved by symbolic constraints or probability-only decoding heuristics. Guided by this perspective, we propose *Reinforced Mode Regulation* (RMR), a lightweight, online state-space intervention that regulates dominant self-reinforcing directions in the
Transformer value cache (implemented as low-rank damping). Across multiple large
language models, RMR substantially reduces mode collapse and enables stable generation at extremely low
entropy rates (down to 0.8 nats/step), whereas standard decoding typically collapses near 2.0 nats/step.
Read the full abstract ▾
Computation and Language
RadLite: Multi-Task LoRA Fine-Tuning of Small Language Models for CPU-Deployable Radiology AI
Pankaj Gupta and Kartik Bose
Large
language models (LLMs) show promise in radiology but their deployment is limited by computational requirements that preclude use in resource-constrained clinical environments. We investigate whether small
language models (SLMs) of 3-4 billion parameters can achieve strong multi-task radiology performance through LoRA fine-tuning, enabling deployment on consumer-grade CPUs. We train Qwen2.5-3B-Instruct and Qwen3-4B on 162K samples spanning 9 radiology tasks - RADS classification across 10 systems, impression generation, temporal comparison, radiology NLI, NER, abnormality detection, N/M staging, and radiology Q&A - compiled from 12 public
datasets. Both models are evaluated on up to 500 held-out test samples per task with standardized metrics. Our key findings are: (1) LoRA fine-tuning dramatically improves performance over zero-shot baselines (RADS accuracy +53%, NLI +60%, N-staging +89%); (2) the two models exhibit complementary strengths - Qwen2.5 excels at structured generation tasks while Qwen3 dominates extractive tasks; (3) a task-outed oracle ensemble combining both models achieves the best performance across all tasks; (4) few-shot prompting with fine-tuned models hurts performance, demonstrating that LoRA adaptation is more effective than in-context learning for specialized domains; and (5) models can be quantized to GGUF format (~1.8-2.4GB) for CPU deployment at 4-8
tokens/second on consumer hardware. Our work demonstrates that small, efficiently fine-tuned models - which we collectively call RadLite - can serve as practical multi-task radiology AI assistants deployable entirely on consumer hardware without GPU requirements. Code and models are available at https://github.com/RadioX-Labs/RadLite
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Rethinking LLM Ensembling from the Perspective of Mixture Models
Jiale Fu, Yuchu Jiang, Peijun Wu et al. (+3 more)
Model ensembling is a well-established technique for improving the performance of machine learning models. Conventionally, this involves averaging the output distributions of multiple models and selecting the most probable label. This idea has been naturally extended to large
language models (LLMs), yielding improved performance but incurring substantial computational cost. This inefficiency stems from directly applying conventional ensemble implementation to LLMs, which require a separate forward pass for each model to explicitly compute the ensemble distribution. In this paper, we propose the Mixture-model-like Ensemble (ME). By reinterpreting the ensemble as a mixture model, ME stochastically selects a single model at each step to generate the next
token, thereby avoiding the need to explicitly compute the full ensemble distribution. ME is mathematically equivalent to sampling from the ensemble distribution, but requires invoking only one model, making it 1.78x-2.68x faster than conventional ensembling. Furthermore, this perspective connects LLM ensembling and
token-level routing methods, suggesting that LLM ensembling is a special case of routing methods. Our findings open new avenues for efficient LLM ensembling and motivate further exploration of
token-level routing strategies for LLMs. Our code is available at https://github.com/Kamichanw/Mixture-model-like-Ensemble.
Read the full abstract ▾
All authors Jiale Fu, Yuchu Jiang, Peijun Wu, Chonghan Liu, Joey Tianyi Zhou, Xu Yang
Computation and Language
Agent Capsules: Quality-Gated Granularity Control for Multi-Agent LLM Pipelines
Aninda Ray
A multi-
agent pipeline with N
agents typically issues N LLM calls per run. Merging
agents into fewer calls (compound execution) promises
token savings, but naively merged calls silently degrade quality through tool loss and
prompt compression. We present
Agent Capsules, an adaptive execution runtime that treats multi-
agent pipeline execution as an optimization problem with empirical quality constraints. The runtime instruments coordination overhead per group, scores composition opportunity, selects among three compound execution strategies, and gates every mode switch on rolling-mean output quality. A controlled negative result confirms that injecting more context into a merged call worsens compression rather than relieving it, so the framework's escalation ladder (standard, then two-phase, then sequential) recovers quality by moving toward per-
agent dispatch rather than by rewriting merged
prompts. On LLM-judged quality, the controller matches a hand-tuned oracle on every measured (model, group, mode)
cell: routing compound whenever the oracle would, and reverting to fine whenever quality would fail the floor, without per-model configuration. Against a hand-crafted LangGraph implementation of a 14-
agent competitive intelligence pipeline,
Agent Capsules uses 51% fewer fine-mode input
tokens and 42% fewer compound-mode input
tokens, at +0.020 and +0.017 quality respectively. Against a DSPy implementation of a 5-
agent due diligence pipeline, the framework uses 19% fewer
tokens than uncompiled DSPy at quality parity, and 68% fewer
tokens than MIPROv2 at +0.052 quality. Even before compound mode fires, the runtime delivers efficiency through automatic policy resolution, cache-aligned
prompts, and
topology-aware context injection, matching both hand-tuned and compile-time baselines without training data or per-pipeline engineering.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
FollowTable: A Benchmark for Instruction-Following Table Retrieval
Rihui Jin, Yuchen Lu, Ting Zhang et al. (+7 more)
Table
Retrieval (TR) has traditionally been formulated as an ad-hoc
retrieval problem, where relevance is primarily determined by topical semantic similarity. With the growing adoption of LLM-based agentic systems, access to structured data is increasingly instruction-driven, where relevance is conditional on explicit content and schema constraints rather than topical similarity alone. We therefore formalize Instruction-Following Table
Retrieval (IFTR), a new task that requires models to jointly satisfy topical relevance and fine-grained instruction constraints. We identify two core challenges in IFTR: (i) sensitivity to content scope, such as inclusion and exclusion constraints, and (ii) awareness of schema-grounded requirements, including column semantics and representation granularity–capabilities largely absent in existing retrievers. To support systematic evaluation, we introduce FollowTable, the first large-scale
benchmark for IFTR, constructed via a taxonomy-driven annotation pipeline. We further propose a new metric, termed the Instruction Responsiveness Score, to evaluate whether
retrieval rankings consistently adapt to user instructions relative to a topic-only baseline. Our results indicate that existing
retrieval models struggle to follow fine-grained instructions over tabular data. In particular, they exhibit systematic biases toward surface-level semantic cues and remain limited in handling schema-grounded constraints, highlighting substantial room for future improvements.
Read the full abstract ▾
All authors Rihui Jin, Yuchen Lu, Ting Zhang, Jun Wang, Kuicai Dong, Zhaocheng Du, Dongping Liu, Gang Wang, Yong Liu, Guilin Qi
Computation and Language
Agentic AI for Substance Use Education: Integrating Regulatory and Scientific Knowledge Sources
Kosar Haghani, Zahra Kolagar and Mohammed Atiquzzaman
The delivery of traditional substance education has remained problematic due to challenges in scalability, personalization, and the currency of information in a rapidly evolving substance use landscape. While artificial intelligence (AI) offers a promising frontier for enhancing educational delivery, its application in providing real-time, authoritative substance use education remains largely underexplored. We built an agentic-based AI web application that combined Drug Enforcement Administration records with peer-reviewed literature in real-time to provide transparent context-sensitive substance use education. The system uses
retrieval-augmented generation with a carefully filtered corpus of 102 documents and dynamic PubMed queries. Document storage was semantically chunked and placed in a vector representation in order to be easily retrieved. We conducted an expert evaluation study in which a panel of five subject matter experts generated 30 domain-specific questions, and two independent raters assessed 90 system interactions (30 primary questions plus two contextual follow-ups each) using a five-point Likert scale across four criteria: factual accuracy, citation quality, contextual coherence, and regulatory appropriateness. Mean ratings ranged from 4.18 to 4.35 across the four criteria (overall category range: 4.05-4.52), with substantial inter-rater agreement (Cohen's kappa = 0.78). These findings suggest that agentic AI architectures integrating authoritative regulatory sources with real-time scientific literature represent a promising direction for scalable, accurate, and verifiable health education delivery, warranting further evaluation through longitudinal user studies.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
ResRL: Boosting LLM Reasoning via Negative Sample Projection Residual Reinforcement Learning
Zihan Lin, Xiaohan Wang, Jie Cao et al. (+6 more)
Reinforcement Learning with Verifiable Rewards (RLVR) enhances
reasoning of Large
Language Models (LLMs) but usually exhibits limited generation diversity due to the over-incentivization of positive rewards. Although methods like Negative Sample Reinforcement (NSR) mitigate this issue by upweighting penalty from negative samples, they may suppress the semantic distributions shared between positive and negative responses. To boost
reasoning ability without losing diversity, this paper proposes negative sample projection Residual Reinforcement Learning (ResRL) that decouples similar semantic distributions among positive and negative responses. We theoretically link Lazy Likelihood Displacement (LLD) to negative-positive head-gradient interference and derive a single-forward proxy that upper-bounds representation
alignment to guide conservative advantage reweighting. ResRL then projects negative-
token hidden representations onto an SVD-based low-rank positive subspace and uses projection residuals to modulate negative gradients, improving
reasoning while preserving diversity and outperforming strong baselines on average across twelve
benchmarks spanning Mathematics, Code,
Agent Tasks, and Function Calling. Notably, ResRL surpasses NSR on mathematical
reasoning by 9.4% in Avg@16 and 7.0% in Pass@128. Code is available at https://github.com/1229095296/ResRL.git.
Read the full abstract ▾
All authors Zihan Lin, Xiaohan Wang, Jie Cao, Jiajun Chai, Li Wang, Xiaodong Lu, Wei Lin, Ran He, Guojun Yin
Computation and Language
Language-free Experience at Expo 2025 Osaka
Michael Paul, Kenji Imamura, Xiaolin Wang et al. (+2 more)
In line with the Global Communication Plan 2025, we have pursued the development of
multilingual translation technologies to realize a
language-barrier-free experience at Expo 2025 Osaka. Our work includes the advancement of simultaneous interpretation systems emphasizing high translation quality and low latency. Key achievements include chunk-based input
segmentation, context-aware translation, and multi-engine machine translation technologies. Through demonstration deployments and collaboration with private companies, our technologies have led to real-world applications, with several services and systems showcased at Expo 2025 Osaka.
Read the full abstract ▾
All authors Michael Paul, Kenji Imamura, Xiaolin Wang, Shohei Higashiyama, Masao Utiyama