Computation and Language
Decoupling Strategy and Execution in Task-Focused Dialogue via Goal-Oriented Preference Optimization
Jingyi Xu, Xingyu Ren, Zhoupeng Shou et al. (+2 more)
Large
language models show potential in task-oriented dialogue systems, yet existing training methods often rely on
token-level likelihood or preference optimization, which poorly align with long-horizon task success. To address this, we propose Goal-Oriented Preference Optimization (GOPO), a hierarchical reinforcement learning framework that decouples strategy planning from response generation via an Expert
Agent and a Customer Service
Agent. The Expert
Agent optimizes multi-turn goal preferences at the dialogue-trajectory level, while the Customer Service
Agent generates responses strictly aligned with the selected strategy. We evaluate GOPO on public
benchmarks and e-commerce customer service
datasets, and introduce Task-focused Sequential Engagement (TSE), a sequence-level metric derived from real e-commerce interaction data. On the Mgshop
dataset, GOPO improves TSE by 7.7% and 10.3% over PPO and Memento, with consistent gains in sequence-level reward and generation quality. Furthermore, a 14B model trained with GOPO achieves 2.7% and 1.5% higher TSE than Qwen-235B and GPT-5.2, respectively. Ablation studies confirm the Expert
Agent's critical role in long-horizon optimization. GOPO demonstrates consistent improvements across other
datasets as well. This work establishes a new paradigm for task-oriented dialogue systems in commercial scenarios, with code and
datasets to be made public.
Read the full abstract ▾
All authors Jingyi Xu, Xingyu Ren, Zhoupeng Shou, Yumeng Zhang, Zhiqiang You
Computation and Language
A Lightweight Explainable Guardrail for Prompt Safety
Md Asiful Islam and Mihai Surdeanu
We propose a lightweight explainable guardrail (LEG) method to detect unsafe
prompts. LEG uses a multi-task learning architecture to jointly learn a
prompt classifier and an explanation classifier, where the latter labels
prompt words that explain the safe/unsafe overall decision. LEG is trained on synthetic explanation data, which is generated using a novel strategy that counteracts the confirmation biases of LLMs. Lastly, LEG's training process uses a novel loss that captures global explanation signals as a weak supervision and combines cross-
entropy and focal losses with uncertainty-based weighting. LEG obtains equivalent or better performance than the state-of-the-art for both
prompt classification and explainability, both in-domain and out-of-domain on three
datasets, despite the fact that its model size is considerably smaller than current approaches.
Read the full abstract ▾
Computation and Language
Building Safe and Deployable Clinical Natural Language Processing under Temporal Leakage Constraints
Ha Na Cho, Sairam Sutari, Alexander Lopez et al. (+2 more)
Clinical natural
language processing (NLP) models have shown promise for supporting hospital discharge planning by leveraging narrative clinical documentation. However, note-based models are particularly vulnerable to temporal and lexical leakage, where documentation artifacts encode future clinical decisions and inflate apparent predictive performance. Such behavior poses substantial risks for real-world deployment, where overconfident or temporally invalid predictions can disrupt clinical workflows and compromise patient safety. This study focuses on system-level design choices required to build safe and deployable clinical NLP under temporal leakage constraints. We present a lightweight
auditing pipeline that integrates interpretability into the model development process to identify and suppress leakage-prone signals prior to final training. Using next-day discharge prediction after elective spine surgery as a case study, we evaluate how
auditing affects predictive behavior, calibration, and safety-relevant trade-offs. Results show that audited models exhibit more conservative and better-calibrated probability estimates, with reduced reliance on discharge-related lexical cues. These findings emphasize that deployment-ready clinical NLP systems should prioritize temporal validity, calibration, and behavioral robustness over optimistic performance.
Read the full abstract ▾
All authors Ha Na Cho, Sairam Sutari, Alexander Lopez, Hansen Bow, Kai Zheng
Computation and Language · Cross-listed from cs.LG
GraphDancer: Training LLMs to Explore and Reason over Graphs via Curriculum Reinforcement Learning
Yuyang Bai, Zhuofeng Li, Ping Nie et al. (+3 more)
Large
language models (LLMs) increasingly rely on external knowledge to improve factuality, yet many knowledge sources on the Web, such as scholarly indexes and product catalogs, are organized as heterogeneous graphs.
Reasoning over such graph-structured knowledge poses two key challenges: (1) navigating structured, schema-defined relations requires precise function calls beyond similarity-based
retrieval, and (2) answering complex questions often demands multi-hop evidence aggregation through iterative information seeking. We study how LLMs can learn these interaction capabilities and transfer them to unseen domains. We propose GraphDancer, a reinforcement learning (RL) framework that teaches LLMs to navigate graphs by interleaving
reasoning and function execution. To improve RL for moderate-sized LLMs, we introduce a graph-aware curriculum that schedules training by the structural difficulty of questions using an easy-to-hard biased sampler. We evaluate GraphDancer on the multi-domain GRBench
benchmark, training only on Academic and Biomedical graphs. Testing covers held-out questions from these domains as well as unseen E-commerce, Literature, and Legal graphs. With either Qwen2.5-3B-Instruct or Qwen3-14B as the backbone, GraphDancer outperforms graph-
agent baselines using the same backbone across all evaluation domains. These results indicate that the learned graph exploration and
reasoning capabilities transfer across domains. On average across unseen domains, GraphDancer with a 14B backbone also outperforms baselines powered by the closed-source models GPT-4.1-mini and GPT-5-nano. Trajectory analyses further show that GraphDancer retrieves answer-bearing evidence more often and achieves higher accuracy when such evidence is found, while substantially reducing invalid-protocol outcomes.
Read the full abstract ▾
All authors Yuyang Bai, Zhuofeng Li, Ping Nie, Jianwen Xie, Yu Wang, Yu Zhang
Computation and Language · Cross-listed from cs.HC
Memento: Towards Proactive Visualization of Everyday Memories with Personal Wearable AR Assistant
Yoonsang Kim, Yalong Yang and Arie E. Kaufman
We introduce Memento, a conversational AR assistant that permanently captures and memorizes user's verbal queries alongside their spatiotemporal and activity contexts. By storing these "
memories," Memento discovers connections between users' recurring interests and the contexts that trigger them. Upon detection of similar or identical spatiotemporal activity, Memento proactively recalls user interests and delivers up-to-date responses through AR, seamlessly integrating AR experience into their daily routine. Unlike prior work, each interaction in Memento is not a transient event, but a connected series of interactions with coherent long–term perspective, tailored to the user's broader
multimodal (visual, spatial, temporal, and
embodied) context. We conduct a preliminary evaluation through user feedbacks with participants of diverse expertise in immersive apps, and explore the value of proactive context-aware AR assistant in everyday settings. We share our findings and challenges in designing a proactive, context-aware AR system.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests
Jingjie Ning, João Coelho, Yibo Kong et al. (+5 more)
LLM-powered search
agents are increasingly being used for multi-step information seeking tasks, yet the IR community lacks empirical understanding of how agentic search sessions unfold and how retrieved evidence is reflected in later queries. This paper presents a large-scale log analysis of agentic search based on 14.44M search requests (3.97M sessions) collected from DeepResearchGym, i.e., an open-source search API accessed by external agentic clients. We sessionize the logs, assign session-level intents and step-wise query-reformulation labels using LLM-based annotation, and propose Context-driven Term Adoption Rate (CTAR) to quantify whether newly introduced query terms are lexically traceable to previously retrieved evidence. Our analyses reveal distinctive behavioral patterns. First, over 90% of multi-turn sessions contain at most ten steps, and 89% of inter-step intervals fall under one minute. Second, behavior varies by intent. Fact-seeking sessions exhibit high repetition that increases over time, while sessions requiring
reasoning sustain broader exploration. Third, query reformulations are often traceable to retrieved evidence across steps. On average, 54% of newly introduced query terms appear in the accumulated evidence context, with additional traceability to earlier steps beyond the most recent
retrieval. These findings provide candidate signals for repetition-aware stopping, intent-adaptive
retrieval budgeting, and explicit cross-step context tracking. We released the anonymized logs, making them available at a public HuggingFace~\chref{https://huggingface.co/
datasets/cx-cmu/deepresearchgym-agentic-search-logs}{repository}.
Read the full abstract ▾
All authors Jingjie Ning, João Coelho, Yibo Kong, Yunfan Long, Bruno Martins, João Magalhães, Jamie Callan, Chenyan Xiong
Computation and Language
What Language Models Know But Don't Say: Non-Generative Prior Extraction for Generalization
Sara Rezaeimanesh, Kundan Thind, Farzan Siddiqui et al. (+1 more)
In domains like medicine and finance, large-scale labeled data is costly and often unavailable, leading to models trained on small
datasets that struggle to generalize to real-world populations. Large
language models contain extensive knowledge from years of research across these domains. We propose LoID (Logit-Informed Distributions), a deterministic method for extracting informative prior distributions for
Bayesian logistic regression by directly accessing their
token-level predictions. Rather than relying on generated text, we probe the model's confidence in opposing semantic directions (positive vs. negative impact) through carefully constructed sentences. By measuring how consistently the LLM favors one direction across diverse phrasings, we extract the strength and reliability of the model's belief about each feature's influence. We evaluate LoID on ten real-world tabular
datasets under synthetic out-of-distribution (OOD) settings characterized by
covariate shift, where the training data represents only a subset of the population. We compare our approach against (1) standard uninformative priors, (2) AutoElicit, a recent method that
prompts LLMs to generate priors via text completions, (3) LLMProcesses, a method that uses LLMs to generate numerical predictions through in-context learning and (4) an oracle-style upper bound derived from fitting logistic regression on the full
dataset. We assess performance using Area Under the Curve (AUC). Across
datasets, LoID significantly improves performance over logistic regression trained on OOD data, recovering up to 59% of the performance gap relative to the oracle model. LoID outperforms AutoElicit and LLMProcessesc on 8 out of 10
datasets, while providing a reproducible and computationally efficient mechanism for integrating LLM knowledge into
Bayesian inference.
Read the full abstract ▾
All authors Sara Rezaeimanesh, Kundan Thind, Farzan Siddiqui, Mohammad M. Ghassemi
Computation and Language
Learning to Ideate for Machine Learning Engineering Agents
Yunxiang Zhang, Kang Zhou, Zhichao Xu et al. (+5 more)
Existing machine learning engineering (MLE)
agents struggle to iteratively optimize their implemented
algorithms for effectiveness. To address this, we introduce MLE-Ideator, a dual-
agent framework that separates ideation from implementation. In our system, an implementation
agent can request strategic help from a dedicated Ideator. We show this approach is effective in two ways. First, in a training-free setup, our framework significantly outperforms implementation-only
agent baselines on MLE-Bench. Second, we demonstrate that the Ideator can be trained with reinforcement learning (RL) to generate more effective ideas. With only 1K training samples from 10 MLE tasks, our RL-trained Qwen3-8B Ideator achieves an 11.5% relative improvement compared to its untrained counterpart and surpasses Claude Sonnet 3.5. These results highlights a promising path toward training strategic AI systems for scientific discovery.
Read the full abstract ▾
All authors Yunxiang Zhang, Kang Zhou, Zhichao Xu, Kiran Ramnath, Yun Zhou, Sangmin Woo, Haibo Ding, Lin Lee Cheong
Computation and Language
LLMs Think with DAGs: Probing the Latent Graph Structure of Reasoning
Tianjun Zhong, Linyang He, Ziyang Li et al. (+1 more)
Large
language models often externalize
reasoning as linear chains, but many
reasoning problems can be more naturally modeled as directed acyclic graphs (DAGs), where intermediate conclusions branch, merge, and are reused. Whether such graph structure is reflected in model internals remains unclear. We introduce
Reasoning DAG Probing to answer this question. We formalize each premise, intermediate conclusion, and final answer as a DAG node, and train lightweight probes to predict node depth, pairwise distance, and adjacency from LLM hidden states. Across logical, mathematical, and code
reasoning benchmarks, we find that DAG structure is meaningfully encoded in LLM representations: it peaks in intermediate layers, with later layers preferentially recover deeper nodes and longer-range
reasoning dependencies, and becomes stronger with model scale. DAG structure also progressively emerges during autoregressive generation in ways that anticipate answer correctness, while steering activations toward the ground-truth DAG structure can convert some originally incorrect generations into correct ones. These results suggest that LLM
reasoning is not purely sequential, but reflects measurable latent graph structure.
Read the full abstract ▾
All authors Tianjun Zhong, Linyang He, Ziyang Li, Nima Mesgarani
Computation and Language · Cross-listed from cs.AI
Intelligence Requires Grounding But Not Embodiment
Marcus Ma and Shrikanth Narayanan
Recent advances in LLMs have reignited scientific debate over whether embodiment is necessary for intelligence. We present the argument that intelligence requires grounding, a phenomenon entailed by embodiment, but not embodiment itself. We define intelligence as the possession of four properties – motivation, predictive ability, understanding of causality, and learning from experience – and argue that each can be achieved by a non-
embodied, grounded
agent. We use this to conclude that grounding, not embodiment, is necessary for intelligence. We then present a thought experiment of an intelligent LLM
agent in a digital environment and address potential counterarguments.
Read the full abstract ▾
Computation and Language
Sequence Repetition Enhances Token Embeddings and Improves Sequence Labeling with Decoder-only Language Models
Matija Luka Kukić, Marko Čuljak, David Dukić et al. (+2 more)
Modern
language models (LMs) are trained in an autoregressive manner, conditioned only on the prefix. In contrast, sequence labeling (SL) tasks assign labels to each individual input
token, naturally benefiting from bidirectional context. This discrepancy has historically led SL to rely on inherently bidirectional encoder-only models. However, the rapid development of decoder-only models has raised the question of whether they can be adapted to SL. While causal mask removal has emerged as a viable technique for adapting decoder-only models to leverage the full context for SL, it requires considerable changes to the base model functionality. In this work, we explore sequence repetition (SR) as a less invasive alternative for enabling bidirectionality in decoder-only models. Through fine-tuning experiments, we show that SR inherently makes decoders bidirectional, improving the quality of
token-level
embeddings and surpassing encoders and unmasked decoders. Contrary to earlier claims, we find that increasing the number of repetitions does not degrade SL performance. Finally, we demonstrate that
embeddings from intermediate layers are highly effective for SR, comparable to those from final layers, while being significantly more efficient to compute. Our findings underscore that SR alleviates the structural limitations of decoders, enabling more efficient and adaptable LMs and broadening their applicability to other
token-level tasks.
Read the full abstract ▾
All authors Matija Luka Kukić, Marko Čuljak, David Dukić, Martin Tutek, Jan Šnajder
Computation and Language · Cross-listed from cs.HC
Status Hierarchies in Language Models
Emilio Barkett
From school playgrounds to corporate boardrooms, status hierarchies – rank orderings based on respect and perceived competence – are universal features of human social organization.
Language models trained on human-generated text inevitably encounter these hierarchical patterns embedded in
language, raising the question of whether they might reproduce such dynamics in multi-
agent settings. This thesis investigates when and how
language models form status hierarchies by adapting Berger et al.'s (1972) expectation states framework. I create multi-
agent scenarios where separate
language model instances complete sentiment classification tasks, are introduced with varying status characteristics (e.g., credentials, expertise), then have opportunities to revise their initial judgments after observing their partner's responses. The dependent variable is deference, the rate at which models shift their ratings toward their partner's position based on status cues rather than task information. Results show that
language models form significant status hierarchies when capability is equal (35 percentage point asymmetry, p < .001), but capability differences dominate status cues, with the most striking effect being that high-status assignments reduce higher-capability models' deference rather than increasing lower-capability models' deference. The implications for AI safety are significant: status-seeking behavior could introduce deceptive strategies, amplify discriminatory biases, and scale across distributed deployments far faster than human hierarchies form organically. This work identifies emergent social behaviors in AI systems and highlights a previously underexplored dimension of the
alignment challenge.
Read the full abstract ▾
Computation and Language
Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculations
Alireza Salemi and Hamed Zamani
Personalization is crucial for aligning Large
Language Model (LLM) outputs with individual user preferences and background knowledge. State-of-the-art solutions are based on
retrieval augmentation, where relevant context from a user profile is retrieved for LLM consumption. These methods deal with a trade-off between exposing retrieved private data to cloud providers and relying on less capable local models. We introduce $P^3$, an interactive framework for high-quality personalization without revealing private profiles to server-side LLMs. In $P^3$, a large server-side model generates a sequence of $k$ draft
tokens based solely on the user query, while a small client-side model, with
retrieval access to the user's private profile, evaluates and modifies these drafts to better reflect user preferences. This process repeats until an end
token is generated. Experiments on LaMP-QA, a recent
benchmark consisting of three personalized question answering
datasets, show that $P^3$ consistently outperforms both non-personalized server-side and personalized client-side baselines, achieving statistically significant improvements of $7.4%$ to $9%$ on average. Importantly, $P^3$ recovers $90.3%$ to $95.7%$ of the utility of a “leaky” upper-bound scenario in which the full profile is exposed to the large server-side model.
Privacy analyses, including linkability and attribute inference attacks, indicate that $P^3$ preserves the
privacy of a non-personalized server-side model, introducing only marginal additional leakage ($1.5%$–$3.5%$) compared to submitting a query without any personal context. Additionally, the framework is efficient for edge deployment, with the client-side model generating only $9.2%$ of the total
tokens. These results demonstrate that $P^3$ provides a practical, effective solution for personalized generation with improved
privacy.
Read the full abstract ▾
Computation and Language
Less is More for RAG: Information Gain Pruning for Generator-Aligned Reranking and Evidence Selection
Zhipeng Song, Yizhi Zhou, Xiangyu Kong et al. (+6 more)
Retrieval-augmented generation (RAG) grounds large
language models with external evidence, but under a limited context budget, the key challenge is deciding which retrieved passages should be injected. We show that
retrieval relevance metrics (e.g., NDCG) correlate weakly with end-to-end QA quality and can even become negatively correlated under multi-passage injection, where redundancy and mild conflicts destabilize generation. We propose Information Gain
Pruning (IGP), a deployment-friendly reranking-and-
pruning module that selects evidence using a generator-aligned utility signal and filters weak or harmful passages before truncation, without changing existing budget interfaces. Across five open-domain QA
benchmarks and multiple retrievers and generators, IGP consistently improves the quality–cost trade-off. In a representative multi-evidence setting, IGP delivers about +12–20% relative improvement in average F1 while reducing final-stage input
tokens by roughly 76–79% compared to retriever-only baselines.
Read the full abstract ▾
All authors Zhipeng Song, Yizhi Zhou, Xiangyu Kong, Jiulong Jiao, Xinrui Bao, Xu You, Xueqing Shi, Yuhang Zhou, Heng Qi
Computation and Language
Revealing the Truth with ConLLM for Detecting Multi-Modal Deepfakes
Gautam Siddharth Kashyap, Harsh Joshi, Niharika Jain et al. (+4 more)
The rapid rise of deepfake technology poses a severe threat to social and political stability by enabling hyper-realistic synthetic media capable of manipulating public perception. However, existing detection methods struggle with two core limitations: (1) modality fragmentation, which leads to poor generalization across diverse and
adversarial deepfake modalities; and (2) shallow inter-modal
reasoning, resulting in limited detection of fine-grained semantic inconsistencies. To address these, we propose ConLLM (Contrastive Learning with Large
Language Models), a hybrid framework for robust
multimodal deepfake detection. ConLLM employs a two-stage architecture: stage 1 uses Pre-Trained Models (PTMs) to extract modality-specific
embeddings; stage 2 aligns these
embeddings via contrastive learning to mitigate modality fragmentation, and refines them using LLM-based
reasoning to address shallow inter-modal
reasoning by capturing semantic inconsistencies. ConLLM demonstrates strong performance across audio,
video, and audio-visual modalities. It reduces audio deepfake EER by up to 50%, improves
video accuracy by up to 8%, and achieves approximately 9% accuracy gains in audio-visual tasks. Ablation studies confirm that PTM-based
embeddings contribute 9%-10% consistent improvements across modalities.
Read the full abstract ▾
All authors Gautam Siddharth Kashyap, Harsh Joshi, Niharika Jain, Ebad Shabbir, Jiechao Gao, Nipun Joshi, Usman Naseem
Computation and Language · Cross-listed from cs.IR
To Case or Not to Case: An Empirical Study in Learned Sparse Retrieval
Emmanouil Georgios Lionis, Jia-Huei Ju, Angelos Nalmpantis et al. (+3 more)
Learned Sparse
Retrieval (LSR) methods construct sparse lexical representations of queries and documents that can be efficiently searched using inverted indexes. Existing LSR approaches have relied almost exclusively on uncased backbone models, whose vocabularies exclude case-sensitive distinctions, thereby reducing vocabulary mismatch. However, the most recent state-of-the-art
language models are only available in cased versions. Despite this shift, the impact of backbone model casing on LSR has not been studied, potentially posing a risk to the viability of the method going forward. To fill this gap, we systematically evaluate paired cased and uncased versions of the same backbone models across multiple
datasets to assess their suitability for LSR. Our findings show that LSR models with cased backbone models by default perform substantially worse than their uncased counterparts; however, this gap can be eliminated by pre-processing the text to lowercase. Moreover, our
token-level analysis reveals that, under lowercasing, cased models almost entirely suppress cased vocabulary items and behave effectively as uncased models, explaining their restored performance. This result broadens the applicability of recent cased models to the LSR setting and facilitates the integration of stronger backbone architectures into sparse
retrieval. The complete code and implementation for this project are available at: https://github.com/lionisakis/Uncased-vs-cased-models-in-LSR
Read the full abstract ▾
All authors Emmanouil Georgios Lionis, Jia-Huei Ju, Angelos Nalmpantis, Casper Thuis, Sean MacAvaney, Andrew Yates
Computation and Language · Cross-listed from cs.LG
PEARL: Prototype-Enhanced Alignment for Label-Efficient Representation Learning with Deployment-Driven Insights from Digital Governance Communication Systems
Ruiyu Zhang, Lin Nie, Wai-Fung Lam et al. (+2 more)
In many deployed systems, new text inputs are handled by retrieving similar past cases, for example when routing and responding to citizen messages in digital governance platforms. When these systems fail, the problem is often not the
language model itself, but that the nearest neighbors in the
embedding space correspond to the wrong cases. Modern machine learning systems increasingly rely on fixed, high-dimensional
embeddings produced by large pretrained models and sentence encoders. In real-world deployments, labels are scarce, domains shift over time, and retraining the base encoder is expensive or infeasible. As a result, downstream performance depends heavily on
embedding geometry. Yet raw
embeddings are often poorly aligned with the local neighborhood structure required by nearest-neighbor
retrieval, similarity search, and lightweight classifiers that operate directly on
embeddings. We propose PEARL (Prototype-Enhanced Aligned Representation Learning), a label-efficient approach that uses limited supervision to softly align
embeddings toward class prototypes. The method reshapes local neighborhood geometry while preserving dimensionality and avoiding aggressive projection or collapse. Its aim is to bridge the gap between purely unsupervised post-processing, which offers limited and inconsistent gains, and fully supervised projections that require substantial labeled data. We evaluate PEARL under controlled label regimes ranging from extreme label scarcity to higher-label settings. In the label-scarce condition, PEARL substantially improves local neighborhood quality, yielding 25.7% gains over raw
embeddings and more than 21.1% gains relative to strong unsupervised post-processing, precisely in the regime where similarity-based systems are most brittle.
Read the full abstract ▾
All authors Ruiyu Zhang, Lin Nie, Wai-Fung Lam, Qihao Wang, Xin Zhao
Computation and Language · Cross-listed from cs.LG
SpatialMath: Spatial Comprehension-Infused Symbolic Reasoning for Mathematical Problem-Solving
Ashutosh Bajpai, Akshat Bhandari, Akshay Nambi et al. (+1 more)
Multimodal Small-to-Medium sized
Language Models (MSLMs) have demonstrated strong capabilities in integrating visual and textual information but still face significant limitations in visual comprehension and mathematical
reasoning, particularly in
geometric problems with diverse levels of visual infusion. Current models struggle to accurately decompose intricate visual inputs and connect perception with structured
reasoning, leading to suboptimal performance. To address these challenges, we propose SpatialMath, a novel Spatial Comprehension-Infused Symbolic
Reasoning Framework designed to integrate spatial representations into structured symbolic
reasoning chains. SpatialMath employs a specialized perception module to extract spatially-grounded representations from visual diagrams, capturing critical
geometric structures and spatial relationships. These representations are then methodically infused into symbolic
reasoning chains, facilitating visual comprehension-aware structured
reasoning. To this end, we introduce MATHVERSE-PLUS, a novel
dataset containing structured visual interpretations and step-by-step
reasoning paths for
vision-intensive mathematical problems. SpatialMath significantly outperforms strong
multimodal baselines, achieving up to 10 percentage points improvement over supervised fine-tuning with data augmentation in
vision-intensive settings. Robustness analysis reveals that enhanced spatial representations directly improve
reasoning accuracy, reinforcing the need for structured perception-to-
reasoning pipelines in MSLMs.
Read the full abstract ▾
All authors Ashutosh Bajpai, Akshat Bhandari, Akshay Nambi, Tanmoy Chakraborty
Computation and Language · Cross-listed from cs.LG
Unintended Memorization of Sensitive Information in Fine-Tuned Language Models
Marton Szep, Jorge Marin Ruiz, Georgios Kaissis et al. (+4 more)
Fine-tuning Large
Language Models (LLMs) on sensitive
datasets carries a substantial risk of unintended memorization and leakage of Personally Identifiable Information (PII), which can violate
privacy regulations and compromise individual safety. In this work, we systematically investigate a critical and underexplored vulnerability: the exposure of PII that appears only in model inputs, not in training targets. Using both synthetic and real-world
datasets, we design controlled extraction probes to quantify unintended PII memorization and study how factors such as
language, PII frequency, task type, and model size influence memorization behavior. We further
benchmark four
privacy-preserving approaches including differential
privacy, machine unlearning, regularization, and preference
alignment, evaluating their trade-offs between
privacy and task performance. Our results show that post-training methods generally provide more consistent
privacy-utility trade-offs, while differential
privacy achieves strong reduction in leakage in specific settings, although it can introduce training instability. These findings highlight the persistent challenge of memorization in fine-tuned LLMs and emphasize the need for robust, scalable
privacy-preserving techniques.
Read the full abstract ▾
All authors Marton Szep, Jorge Marin Ruiz, Georgios Kaissis, Paulina Seidl, Rüdiger von Eisenhart-Rothe, Florian Hinterwimmer, Daniel Rueckert
Computation and Language
Clustering-driven Memory Compression for On-device Large Language Models
Ondrej Bohdal, Pramit Saha, Umberto Michieli et al. (+2 more)
Large
language models (LLMs) often rely on user-specific
memories distilled from past interactions to enable personalized generation. A common practice is to concatenate these
memories with the input
prompt, but this approach quickly exhausts the limited context available in on-device LLMs. Compressing
memories by averaging can mitigate context growth, yet it frequently harms performance due to semantic conflicts across heterogeneous
memories. In this work, we introduce a clustering-based
memory compression strategy that balances context efficiency and personalization quality. Our method groups
memories by similarity and merges them within clusters prior to concatenation, thereby preserving coherence while reducing redundancy. Experiments demonstrate that our approach substantially lowers the number of
memory tokens while outperforming baseline strategies such as naive averaging or direct concatenation. Furthermore, for a fixed context budget, clustering-driven merging yields more compact
memory representations and consistently enhances generation quality.
Read the full abstract ▾
All authors Ondrej Bohdal, Pramit Saha, Umberto Michieli, Mete Ozay, Taha Ceritli
Computation and Language · Cross-listed from cs.LG
Data-driven Clustering and Merging of Adapters for On-device Large Language Models
Ondrej Bohdal, Taha Ceritli, Mete Ozay et al. (+4 more)
On-device large
language models commonly employ task-specific adapters (e.g., LoRAs) to deliver strong performance on downstream tasks. While storing all available adapters is impractical due to
memory constraints, mobile devices typically have sufficient capacity to store a limited number of these parameters. This raises a critical challenge: how to select representative adapters that generalize well across multiple tasks - a problem that remains unexplored in existing literature. We propose a novel method D2C for adapter clustering that leverages minimal task-specific examples (e.g., 10 per task) and employs an iterative optimization process to refine cluster assignments. The adapters within each cluster are merged, creating multi-task adapters deployable on resource-constrained devices. Experimental results demonstrate that our method effectively boosts performance for considered storage budgets.
Read the full abstract ▾
All authors Ondrej Bohdal, Taha Ceritli, Mete Ozay, Jijoong Moon, Kyeng-Hun Lee, Hyeonmok Ko, Umberto Michieli
Computation and Language · Cross-listed from cs.CY
The 17% Gap: Quantifying Epistemic Decay in AI-Assisted Survey Papers
H. Kemal İlter
The adoption of Large
Language Models (LLMs) in scientific writing promises efficiency but risks introducing informational
entropy. While "hallucinated papers" are a known artifact, the systematic degradation of valid citation chains remains unquantified. We conducted a forensic audit of 50 recent survey papers in Artificial Intelligence (N=5,514 citations) published between September 2024 and January 2026. We utilized a hybrid verification pipeline combining DOI resolution, Crossref metadata analysis, Semantic Scholar queries, and fuzzy text matching to distinguish between formatting errors ("Sloppiness") and verifiable non-existence ("Phantoms). We detect a persistent 17.0% Phantom Rate – citations that cannot be resolved to any digital object despite aggressive forensic recovery. Diagnostic categorization reveals three distinct failure modes: pure
hallucinations (5.1%), hallucinated identifiers with valid titles (16.4%), and parsing-induced matching failures (78.5%). Longitudinal analysis reveals a flat trend (+0.07 pp/month), suggesting that high-
entropy citation practices have stabilized as an endemic feature of the field. The scientific citation graph in AI survey literature exhibits "link rot" at scale. This suggests a mechanism where AI tools act as "lazy research assistants," retrieving correct titles but hallucinating metadata, thereby severing the digital chain of custody required for reproducible science.
Read the full abstract ▾