Computation and Language
CORAL: An LLM-Native Harness for Production Recommender Systems
Muhammad Rafay Azhar, Yuhang Zhou, Gilbert Jiang et al. (+7 more)
Production recommender systems shape what billions of people see, and sustaining their performance requires continual optimization: as content, user behavior, and upstream models shift, the choices governing
retrieval, ranking, and serving must be revisited. Traditionally, human engineers test such changes through online experiments–a slow, reactive process limited by engineering effort, leaving parts of the system unrevised as conditions change. Although large
language models have been applied to ranking, user modeling, and offline model development, few systems place an
agent in a continual closed loop that acts on a live recommender and learns from the measured effects of its decisions. We present CORAL (Constraint-Optimized Recommender via an Agentic Loop), an LLM-native harness that closes this loop: each cycle, the
agent observes operating signals, reasons over a
memory of past decisions and outcomes, and invokes tools–including a numerical optimizer that keeps changes within a fixed operating budget–to reconfigure the recommender, with measured outcomes informing the next cycle. We formulate this as a partially observed, non-stationary, constrained optimization problem in which the policy improves in context, without parameter updates, from its prior actions. Across two large-scale social platforms, evaluated with A/B experiments, the same harness improves engagement at no additional serving cost on one and reduces serving cost without degrading engagement on the other, spanning the engagement-efficiency frontier. Performance improves as the loop iterates, suggesting that a single agentic loop can automate continual optimization work traditionally performed by human
algorithm engineers under explicit guardrails.
Read the full abstract ▾
All authors Muhammad Rafay Azhar, Yuhang Zhou, Gilbert Jiang, Yuchen Wang, Rahul Sharma, Matthew DeSousa, Jiayi Liu, Xin Guo, Lizhu Zhang, Xiangjun Fan
Computation and Language · Cross-listed from cs.AI
Door-in-the-Face Requests and Refusal Behaviour in Large Language Models
Til Jordan
Does the door-in-the-face technique work on
language models? In humans, a large request that is refused makes a smaller follow-up request more likely to be granted. We test this on nine production models from three providers: each model refuses a large request, then receives a smaller version of the same request, and we compare its compliance with asking directly. The answer depends on the model. On Anthropic's frontier models the technique works: Opus 5 answers the smaller request 65.8% of the time after refusing the larger one, against 29.3% when asked directly. On the frontier models of OpenAI and Google, and on Haiku 4.5, it backfires, lowering compliance by 15.5 to 23.0 points. A control locates the effect: a refused large request on an unrelated topic does less than the related one on all nine models, so the concession itself matters everywhere, while the reaction to having just refused something differs by model family. The technique does not transfer to refusals drawn from public
benchmarks. What decides whether a retreat can work is what the request asks for: rewriting 265 refused requests for usable instructions into requests for explanations of the same topic removed the refusal in 263 cases. Human influence techniques port to
language models one model family at a time.
Read the full abstract ▾
Computation and Language
Trace as State: Reasoning Traces as Conditional States for Long-Context Transformers
Xu Zou and Jie Tang
Transformers process information causally, but long-context
reasoning may depend on task state discovered only later. We formalize this mismatch through conditional state update tasks. For causal state update processors, providing the condition first can require exponentially less
memory in the worst case than providing it last. Motivated by this principle, we introduce Trace as State. We use collected
reasoning traces as a textual proxy for task state and place it before the long-context block on a fresh pass, allowing information derived previously to guide rereading. We conduct extensive experiments on Trace as State and Trace Append, a matched control that uses the same task state proxy but put it after the context. Across three models and three long-context
datasets, Trace as State outperforms Trace Append in 26 of 27 reported combinations of model, task, and metric. On GraphWalks Parents, exact match lifts DeepSeek V4 Pro Preview from 29.2% on the initial pass and 43.0% with Trace Appendto 81.8% with Trace as State, and from 66.4% and 83.2% to 100.0% for GLM-5.2. These results show that placing traces before the context can improve long-context
reasoning while retaining the causal
transformer structure.
Read the full abstract ▾
Computation and Language
DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models
Kushagra Bhushan, Meghanadh Pulivarthi, Sai Krishna Reddy Sathi et al. (+7 more)
RAG has become the de facto method for incorporating new, corpus-specific knowledge into an instruction following LLM (Instruct LLM). Although RAG-based prompting improves factual grounding, it fails when
retrieval is incorrect or incomplete, leading to
hallucinations. Finetuning methods such as RAFT and PA-RAG enhance RAG by injecting new knowledge into the model's parameters, but require generating a massive amount of synthetic QA that covers the entire corpus. Extended Pre-Training (EPT) on the text corpus avoids the need for comprehensive synthetic data generation but compromises an Instruct LLM's instruction-following capabilities, necessitating instruction fine-tuning (IFT) after pre-training. However, IFT is costly and may be infeasible due to the unavailability of an instruction-tuning corpus. In this work, we propose DKL-Decoupled Knowledge Learning for Instruction-Tuned
Language Models. Instead of doing EPT on the Instruct LLM, DKL performs EPT on its corresponding base LLM to infuse new knowledge. These knowledge infused weights are then merged with the Instruct LLM, imparting new knowledge without affecting their instruction-following capabilities. DKL is a lightweight method that avoids expensive instruction fine-tuning and relies on model merging to infuse the new knowledge into the Instruct LLM without destroying its instruction following capabilities. Empirical results show that DKL improves RAG accuracy from 54.17 to 79.26 on
retrieval failure cases, while outperforming prior approaches with substantially less training data.
Read the full abstract ▾
All authors Kushagra Bhushan, Meghanadh Pulivarthi, Sai Krishna Reddy Sathi, Gaurav Pandey, Sonam Gupta, Vineet Kumar, Jaydeep Sen, Yatin Nandwani, Sachindra Joshi, Dinesh Raghu
Computation and Language
From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs
Urja Pawar, Rajitha Ramanayake, Owen O'Neill et al. (+5 more)
When LLMs support public-facing or high-stakes workflows, missed fabrications can harm users and institutions, while false alarms consume limited human-review capacity. When no trusted context or reference document is available, we study two signals accessible through black-box model APIs: semantic
entropy, which measures disagreement among sampled response meanings, and uncertainty derived from
token log-probabilities. Their failure modes can be complementary: semantic
entropy becomes uninformative when responses form one semantic cluster, while
token uncertainty can miss consistently confident errors. We extend
token-based uncertainty detection by aggregating
token-level signals across sampled responses through our TopK method, evaluate the hybrid CoCoA method, which combines target-response uncertainty with semantic dissimilarity, and propose and study two supervised methods: Gated, which routes single-cluster cases to an aggregated-
token-feature classifier, and Stacked, which learns jointly from semantic uncertainty and broader
token features. We evaluate seven
benchmarks, including five public
benchmarks (four text
datasets and
multimodal handwritten-cheque extraction) and two constructed
benchmarks (Financial Summaries and Long-Text QA), using four
language models. In our evaluation across models and
datasets, Stacked gave the best performance in nearly half of the cases, while TopK and CoCoA remain competitive without supervised training labels, although their thresholds require careful calibration. No method is universally strongest. We therefore evaluate performance at false-positive-rate budgets from 1% to 15%, assess their sensitivity to generation and calibration choices, and examine variation across
dataset characteristics.
Read the full abstract ▾
All authors Urja Pawar, Rajitha Ramanayake, Owen O'Neill, Nabeel Kemal, Abhishek Mandal, Houssem Chatbri, Christopher Martin, Vadim Pertsovskiy
Computation and Language
oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions
Haoqiang Guo, Xuyi Chen, Bo Ke et al. (+5 more)
Hyper-Connections (HC) replace the single residual stream of a
Transformer with $n$ parallel ones, mixing them at every layer with a learned $n \times n$ residual
matrix. Leaving that
matrix unconstrained places no limit on the factor by which the mixing step rescales the residual streams, and that factor compounds across layers, which destabilizes training.
Manifold-constrained Hyper-Connections (mHC) address this by restricting the
matrix to the doubly
stochastic matrices. That caps the factor at one, so the mixing can no longer amplify any direction, but nothing bounds it from below. We prove that inside this set the mixing step can reduce the norm of the residual streams only by shrinking the differences between the streams, while their mean is left unchanged; and since the reduction accumulates over layers, the streams grow more alike and their diversity is spent with depth. We therefore propose Orthogonal Hyper-Connections (oHC), restricting the residual
matrix to the rotation group $SO(n)$, so that the mixing step can neither amplify nor attenuate the residual streams in any direction, which keeps training stable and no longer forces the differences between the streams to contract. Specifically, at the four streams used by recent HC models we parameterize the group in closed form by a pair of unit quaternions, which adds no parameters, replaces the iterative projection with a fixed pattern of signed additions, and can be constructed faster than mHC. We evaluate oHC across a comprehensive set of downstream tasks, where it outperforms the single-stream residual baseline, mHC and iHC, which fixes the residual
matrix to the identity.
Read the full abstract ▾
All authors Haoqiang Guo, Xuyi Chen, Bo Ke, Yishu Lei, Ziyang Xu, Shikun Feng, Ximen, Wenhan Luo
Computation and Language
WinoQueer-NL: Assessing Bias in Dutch Language Models toward LGBTQ+ Identities
Jiska Beuk and Gerasimos Spanakis
While English
language models have been widely examined for anti-queer bias, Dutch models remain understudied. To address this gap, we developed a culturally and linguistically adapted Dutch
dataset based on the English WinoQueer
benchmark, containing pairs of stereotypical and counter-stereotypical sentences. To validate and expand it, we conducted an online survey with 43 Dutch queer participants, confirming 145 of 171 stereotypes as culturally relevant and identifying 22 new biases through free-text responses. The final released
dataset, comprising 42,906 sentences, was evaluated using a range of Dutch-specific and
multilingual models, including both masked
language models (MLMs) and autoregressive
language models (ARLMs), with bias measured via a score comparing log-likelihoods of stereotypical versus counter-stereotypical sentences. While the mean bias score across models appeared neutral (~50%), closer analysis revealed significant disparities: some models favored stereotypical sentences up to 97% of the time for transgender identities, but only 6% of the time for gay-related pairs, with transgender and non-binary identities consistently receiving the highest bias scores. Our findings highlight the importance of culturally grounded
datasets for evaluating and mitigating biases that disproportionately impact marginalized groups in Dutch
language models.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Loom: Weaving Diagnostic Strands into Free-Text Consensus via Embedding-Space Reweighting
Ron Begleiter, Katya Egert Berg, Gilad Saban et al. (+1 more)
Aggregating noisy, conflicting textual hypotheses into a reliable consensus is a fundamental challenge when deploying NLP systems in real-world industrial settings. While monolithic Large
Language Model (LLM)
agents offer unbounded expressivity for tasks like Root Cause Analysis (RCA), they suffer from context limits, compounding
hallucinations, and prohibitive inference latency. Traditional weak supervision offers statistical rigor but is mathematically restricted to discrete classes. We present Loom, a generative consensus framework deployed for real-world RCA that bridges these paradigms. Loom aggregates open-form hypotheses emitted by modular heuristics (diagnostic templates dynamically populated with episode-specific entities, times, and metrics) by projecting them into a continuous
embedding space, and resolves conflicting signals with an iterative centroid-based reweighting
algorithm. The resulting consensus weights ground a single lightweight LLM synthesis step. Evaluated on the OpenRCA
benchmark, Loom occupies the accuracy–efficiency Pareto frontier: it matches a state-of-the-art autonomous
agent on Bank and
Market-2 and trails on
Market-1 and Telecom, while using a single LLM call per incident on all four
datasets ($\sim$26$\times$ faster; $\sim$33$\times$ with an 8B-parameter synthesizer). We discuss our deployment experience, highlighting lessons learned regarding the trade-offs between agentic depth and inference latency, negative results in redundancy detection, and how deterministic consensus fosters trust among Subject Matter Experts~(SMEs).
Read the full abstract ▾
All authors Ron Begleiter, Katya Egert Berg, Gilad Saban, Gil Shabat
Computation and Language
TaRA: Training-Aware Low-Rank Adaptation Initialization
Taehyeon Kim and Eunhyeok Park
Low-Rank Adaptation (LoRA) has become a de facto standard for parameter-efficient fine-tuning (PEFT), yet its performance is highly sensitive to initialization due to the information bottleneck imposed by low-rank decomposition. Existing approaches attempt to construct high-quality LoRA initializations by exploiting principal components of pretrained weights, activations, or gradients. However, these methods do not directly account for the training dynamics of the full-rank model. In this paper, we propose Training-aware Low-Rank Adaptation Initialization (TaRA), a method that initializes LoRA such that the gradients induced by the low-rank factors closely approximate the gradient of the corresponding full-rank weight
matrix. Derived from a mathematical formulation, TaRA improves gradient fidelity at the start of training while introducing negligible computational overhead. Across diverse and challenging fine-tuning tasks, TaRA consistently outperforms prior state-of-the-art methods, establishing a simple, robust, and scalable solution for effective LoRA initialization.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
Scalable Direction-Following TTS via Voice Impression-Guided Pseudo Triplet Construction
Kenichi Fujita and Yusuke Ijima
Voice actors often re-read the same script while modifying their delivery in response to performance directions. We study this setting as direction-following TTS, where a system generates a new utterance that reflects a given direction relative to a reference utterance while preserving speaker identity and linguistic content. A key challenge is the lack of training data capturing such relative modifications. To address this, we propose a scalable pseudo-triplet construction pipeline that generates~(reference utterance, direction text, modified utterance) triplets. It generates controlled style variations using an impression-controllable TTS model and uses an LLM to produce natural
language directions from estimated impression differences. Experimental results demonstrate that pseudo-triplets alone enable stable speaker-preserving modification, and that combining pseudo and recorded data further improves direction
alignment while maintaining speaker similarity. Audio examples are available on our demo page https://ntt-hilab-gensp.github.io/IS2026pseudo/
Read the full abstract ▾
Computation and Language
Predictors of Loneliness in Older Adults Using Multimodal Analysis of Speech and Language
Vinmay Khandode, Sai Karthik Kosuri, Neil K. R. Sehgal et al. (+6 more)
Loneliness is a critical public health issue among older adults, linked to higher risks of depression, cognitive decline, and mortality. Scalable, objective methods for its detection remain limited, particularly in natural conversational contexts. We analyzed
speech and
language markers of loneliness in 310 older adults using semi-structured telephone interviews to help understand how they process feeling lonely and how their
language differs at different levels of feeling loneliness. Our
multimodal framework combined linguistic features (psycholinguistic dictionaries, n-grams, and topic models) with acoustic features (pitch, tone, loudness) to examine associations with self-reported loneliness scores. Both predefined and data-driven methods captured patterns in verbal content and vocal delivery. Higher loneliness was associated with negations(r = 0.11), negative tone(r = 0.12), and conflict-related
language. Lower loneliness was linked to social references(r = -0.18), motivational drives(r = -0.11), and emotional richness in
speech(r = -0.12). We also found that the
multimodal model (r = 0.298) outperforms the text-only and audio-only models. Findings suggest that loneliness manifests through both linguistic and acoustic cues, supporting the potential of
speech-based analysis in psychological assessments and as an early indicator of emotional loneliness when used alongside existing assessments, rather than as standalone diagnostic tools.
Read the full abstract ▾
All authors Vinmay Khandode, Sai Karthik Kosuri, Neil K. R. Sehgal, Adam Greene, Elif Alpoge, Elana Duffy, Matthew Lee Smith, Thomas K.M. Cudjoe, Sharath Chandra Guntuku
Computation and Language
When Persona Attributes Improve Population Alignment in Large Language Models
Leon Fröhling, Jens Rupprecht, Markus Strohmaier et al. (+1 more)
Large
Language Models (LLMs) are increasingly used to predict the responses of human participants in survey panels. Towards that goal, persona prompting has recently emerged as a technique to inform and align large pretrained
language models. Persona prompting refers to the practice of using short textual descriptions of 'personas' in
prompts to steer the LLM's generations. Personas describe individuals through different attributes such as their socio-demographics, attitudes, or behaviors, with the aim of aligning LLMs to produce responses that correlate with the corresponding human responses. Yet, recent work has produced mixed and partly conflicting results of persona prompting without clear patterns of success and failure. Among the few consistent findings is that the selection of persona attributes matters, and that using more attributes does not necessarily lead to better performance. It remains unclear how different attribute selection methods perform and how to choose among them. In this paper, we propose that observed human response variation of a survey question is a potential explanation for the mixed performance observed so far. In addition, we compare the performance of persona prompting associated with different methods for selecting persona attributes. We evaluate these methods on four different (general) social surveys across two countries, six LLMs, and twenty prediction tasks per survey. Our work helps to identify when persona prompting can be expected to be useful in survey prediction tasks, and provides new insights on the effectiveness of different attribute selection methods for LLM-based survey prediction using persona prompting.
Read the full abstract ▾
All authors Leon Fröhling, Jens Rupprecht, Markus Strohmaier, Claudia Wagner
Computation and Language
Debias-SparseGPT: Bias-Aware Pruning for Large Language Models
Irina Proskurina, Guillaume Metzler, Antoine Gourru et al. (+1 more)
Model compression techniques such as
pruning and quantization facilitate the efficient deployment and acceleration of Large
Language Models (LLMs). However, recent studies show that weight sparsification methods, such as SparseGPT, can amplify existing biases in models, with outputs varying significantly depending on persona cues in the
prompt. In this paper, we introduce Debias-SparseGPT, a post-training
pruning method incorporating representational debiasing using a second-order term defined over demographically contrasting inputs. We perform empirical validation of our method over a wide range of generative LLMs. Across models and sparsity regimes (25%, 50%, and structured 2:4 sparsity), Debias-SparseGPT consistently reduces
pruning-induced bias compared to SparseGPT while preserving model perplexity and zero-shot accuracy. Under the most restrictive 2:4 structured sparsity pattern, which most aggressively degrades model quality, augmenting the calibration set with long-context, content-rich examples further improves both downstream performance and
fairness. Overall, Debias-SparseGPT advances the bias-performance trade-off while preserving the computational efficiency of sparse models.
Read the full abstract ▾
All authors Irina Proskurina, Guillaume Metzler, Antoine Gourru, Julien Velcin
Computation and Language · Cross-listed from cs.IR
ViSAR: Training-Free Adaptive-$k$ Retrieval for Visual Document Question Answering
Adrien Mialland, Marc Plantevit, Julien Gallois et al. (+1 more)
Document Visual Question Answering (DocVQA) often leverages
Retrieval-Augmented Generation (RAG), where late-interaction encoders are commonly used to identify document pages relevant to a user query, before answer generation by a Large
Vision-
Language Model (LVLM). Existing approaches typically retrieve a fixed top-$k$ number of pages regardless of query complexity, which increases LVLM latency and may degrade answer accuracy. We introduce ViSAR (Visual Semantic Activation
Retrieval), a training-free adaptive-$k$
retrieval method for late-interaction visual document
retrieval. ViSAR operates directly in the
embedding space to construct a query-conditioned page-level similarity
matrix that highlights query-relevant semantics and dynamically determines the number of pages to retrieve. Across multiple encoders and LVLMs, ViSAR retrieves compact, query-adapted page sets that reduce RAG latency by up to 58.7%, while maintaining or improving answer accuracy compared with fixed top-$k$ and adaptive
retrieval heuristics. Furthermore, we show that the similarity
matrix structure correlates with answer accuracy, suggesting future directions for
retrieval quality-aware document understanding.
Read the full abstract ▾
All authors Adrien Mialland, Marc Plantevit, Julien Gallois, Céline Robardet
Computation and Language
How LLMs Build Fictional Worlds: Setting and Narrative Space in AI-Generated Creative Storytelling
Katrin Rohrbacher, Björn Nieth, Emmanuelle Salin et al. (+2 more)
In this paper, we analyze how Large
Language Models (LLMs) employ worldbuilding strategies, focusing on setting as one measurable dimension of storyworld construction. We compare 1,000 AI-generated stories per model in English and German with human-authored fiction from Project Gutenberg. Building on prior work, we operationalize setting through five types of narrative space: "action", "perceived," "visual," "descriptive" and "no space", identified using fine-tuned BERT classifiers for German and English. We generate narratives using GPT 4.1, LlaMA 3.3, Mistral 3.2, and Gemma 3 and compare their spatial distributions to a human-authored baseline. We find that human-authored texts predominantly employ "action space," grounding narratives in
embodied character-environment interaction, whereas LLMs systematically overproduce "perceived space," emphasizing atmosphere and affect. This divergence remains stable across narrative time. Overall, our findings show that LLMs exhibit worldbuilding patterns that differ consistently from human-authored fiction in ways that are both model-specific and
language-sensitive.
Read the full abstract ▾
All authors Katrin Rohrbacher, Björn Nieth, Emmanuelle Salin, Bjoern Eskofier, Michaela Mahlberg
Computation and Language
PragAlign: Feedback-Guided Pragmatic Alignment for Controlled Synthetic Dialogue Generation
Smitha Muthya Sudheendra and Jaideep Srivastava
Synthetic dialogue generation can support research in
privacy-restricted service settings, but generated conversations must preserve communicative intent, affective meaning, and natural dialogue flow. We introduce PragAlign, a feedback-guided framework for controlled synthetic dialogue generation conditioned on service context, target intent, and target
emotion, with auxiliary trait-style controls. PragAlign uses a generate–evaluate–revise loop in which an LLM-based evaluator scores intent
alignment,
emotion alignment, coherence, fluency, and aggregate quality, then provides criterion-specific feedback for up to three refinement rounds. On 800 matched dialogue specifications, PragAlign achieves 99.50% evaluator-defined acceptance, compared with 72.25% for one-shot generation and 95.88% for repeated generation without structured feedback. This indicates that repeated attempts account for much of the gain over one-shot generation, while structured feedback primarily improves last-mile multi-constraint satisfaction rather than broad average quality. Refinement gains are concentrated in
emotion alignment, which is also the dominant failure mode in ablations. A separate human evaluation of 1,200 generated dialogues shows that intent expression and dialogue flow are highly recognizable to annotators, while
emotion appropriateness is less stable and more subjective. These results support PragAlign as a quality-control framework for improving evaluator-defined communicative constraint satisfaction, while showing that affective realization and independent human-perceived quality remain open challenges.
Read the full abstract ▾
Computation and Language
Learning to Fuse LLMs with Ontology Rankers for Rare-Disease Diagnosis
Zhaoyang Jiang, Zhizhong Fu, Yunsoo Kim et al. (+5 more)
Ontology rankers remain useful for rare-disease diagnosis because each candidate can be traced to matched patient phenotypes. Large
language models (LLMs) can generate differential diagnoses from the same patient description, but their predictions lack an equally clear evidence trail. Rather than asking which system should replace the other, we ask whether an LLM can improve the ranker without giving up its evidence. Our behavior-based fusion model examines the two ranked lists, their agreement, and the ontology support behind each candidate, and learns how much to rely on each system for the individual case. Before comparison, we remove a documented test-set leakage pathway caused by
benchmark cases and ontology annotations being derived from the same publications. Across eight open LLMs, fusion improves Phenomizer Recall@1 by 7.86 percentage points on Phenopacket Store and 20.18 points on RAMEDIS. When paired with DeepSeek-V4-Flash through an API, a fusion model trained only on the other LLMs improves Recall@1 from 0.1657 to 0.2176, a 5.19-point gain, without retraining. For 90.8% of correct fused diagnoses, the disease retains candidate-level ontology evidence that can be inspected. These results show that LLMs can strengthen an established diagnostic tool without discarding the structured evidence that makes it useful.
Read the full abstract ▾
All authors Zhaoyang Jiang, Zhizhong Fu, Yunsoo Kim, Zicheng Li, Xuanqi Peng, Fei Teng, Jiacong Mi, Honghan Wu
Computation and Language · Cross-listed from cs.LG
Scalable Kronecker-Fisher Approximation: Efficient Hessian Analysis for Billion-Parameter Language Models Compression
Viacheslav Yusupov, Daria Cherniuk and Evgeny Frolov
In this paper, we propose a scalable Kronecker-based approximation that captures cross-layer interactions without storing the entire Fisher
matrix, enabling practical Hessian analysis for billion-parameter networks where full computation is infeasible. Our approach reveals consistent vulnerability patterns: value projection layers exhibit the highest sensitivity and strongest cross-layer correlations across multiple model families, while other components exhibit architecture-specific behaviors. Through extensive experiments on quantization, sparsification, inter-layer corruption, and post-corruption fine-tuning, we demonstrate that our approximation strongly correlates with both performance degradation and recovery. Our framework provides a practical, theoretically grounded tool for identifying fragile components in large models, opening new avenues for guided compression and optimization strategies, such as mixed-precision allocation, layer-wise sparsity, and adaptive low-rank decomposition across layers and even individual weight groups.
Read the full abstract ▾
Computation and Language
When Decodability Is Not Enough: Logical Validity Representations, Behavioral Dissociation, and Causal Tests in Language Models
Smitha Muthya Sudheendra and Jaideep Srivastava
Large
language models can look capable of logical
reasoning, but correct or incorrect answers alone tell us little about what the model represents internally. We study logical verification in five open-weight
transformer models using matched valid–invalid premise–claim pairs that vary across inference families, semantic domains, templates, and difficulty levels. Despite near-chance behavioral performance, logical validity is often almost perfectly decodable from hidden states and remains strongly decodable under held-out templates, domains, and inference families. Validity also remains highly decodable on behaviorally incorrect examples in the conditions where correctness-conditioned evaluation is well defined. At the same time, exhaustive leave-one-out tests reveal clear limits to this generalization, and interventions along probe-derived validity directions have only weak, nonspecific effects compared with random controls. Our results suggest that representing validity, expressing it in behavior, and using it causally are distinct. Validity related information can be strongly decodable from a model's hidden states without being reliably expressed in its output.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
UTP-Bench: Uncertainty-aware Travel Planning Benchmark
Etcharla Revanth Rao, Priyanshu Karmakar, Shubhojit Mallick et al. (+3 more)
Large
Language Models (LLMs) have recently demonstrated strong capabilities in automated travel itinerary generation. However, real- world travel planning is inherently uncertain: transportation delays, crowd fluctuations, and unexpected
stochastic delays frequently inval- idate otherwise feasible schedules. Existing
benchmarks like TravelPlanner and TripCraft assume deterministic environments, evaluating only static constraint satisfaction and ignoring whether generated plans remain robust when such uncertainties arise. To address this limitation, we introduce UTP-Bench1 , a large-scale
benchmark for uncertainty-aware travel planning. The
dataset integrates real-world travel data spanning 504 cities of India, including attractions, restau- rants, accommodations, and multi-modal trans- portation networks. To model realistic disrup- tions, UTP-Bench incorporates empirical delay distributions and crowd-density patterns col- lected from major cities, enabling evaluation of travel plans under
stochastic conditions. We further propose three evaluation metrics, namely Buffer Adequacy Score (BAS), Crowd- Aware Timing Score (CATS), and Transport Delay Absorption Score (TDAS), which quan- tify the ability of generated itineraries to main- tain robustness against transit delays and crowd variability. Experiments with state-of-the-art LLMs like GPT-5, Qwen3, Mistral and Phi-4 re- veal substantial gaps between model-generated and human-authored plans, particularly in tem- poral buffering, delay-aware transportation scheduling, and crowd-sensitive planning.
Read the full abstract ▾
All authors Etcharla Revanth Rao, Priyanshu Karmakar, Shubhojit Mallick, Manish Gupta, Shreya Ghosh, Abhik Jana
Computation and Language
Before the Script, Set the Stage: How Worldview Simulation Amplifies Psychologically Grounded Persuasion in Multi-Turn Jailbreaking
Siyu Chen, Haoran Wang, Xiaojian Li et al. (+3 more)
Multi-turn jailbreak attacks demonstrate that harmful intent can be distributed across dialogue, yet existing methods obscure what conversational mechanisms drive vulnerability. We introduce BLUEPRINT, a safety-evaluation framework separating a factorized social-influence strategy space from WORLDVIEWSIM, a cross-turn situational context module. Monte Carlo Tree Search optimizes turn-level combinations of 18 theory-grounded influence factors across a four-turn trajectory. Across six frontier models, BLUEPRINT achieves near-ceiling ASR on major open-weight and proprietary models, while requiring the fewest average queries (2.46). The resulting trajectories further reveal model-specific vulnerability among resistant targets: each responds to distinct influence factors and strategy transitions, yet all share a common recovery pathway-shifting toward concrete, executable task framing consistently escapes hard-refusal states. Ablations confirm operational cues matter most: making requests actionable has the largest impact, gain framing is unusually potent, and some legitimacy appeals can backfire. These findings suggest robust multi-turn safety requires monitoring not only harmful content, but also how dialogue state makes unsafe requests appear concrete and locally executable.
Read the full abstract ▾
All authors Siyu Chen, Haoran Wang, Xiaojian Li, Yao Huang, Yinpeng Dong, Wei Xu
Computation and Language
Improving Health Literacy through Lay Summarization of Radiological Reports: An Evaluation of BioNER and Retrieval-Augmented Generation
Egecan Çelik Evgin, İlknur Karadeniz and Olcay Taner Y{\i}ld{\i}z
Radiology reports are written primarily for clinicians, and their specialized terminology often makes them difficult for patients to interpret. As a result, many patients turn to publicly available Large
Language Models (LLMs) to help explain their reports, despite well-documented risks of factual inaccuracies and
hallucinations. Automated lay-summary generation has emerged as a promising alternative, yet the effectiveness of
retrieval-enhanced and clinically informed approaches for radiology-specific communication remains underexplored. This study investigates the extent to which
Retrieval-Augmented Generation (RAG) and Named Entity Recognition (NER) improve the quality, factual consistency, and readability of automatically generated lay summaries compared with standard LLM-based generation. We develop a framework combining NER-based extraction of clinically relevant findings with a RAG mechanism for contextual grounding, evaluated across few-shot and fine-tuned variants of two models (Qwen, BioBART). Results show that NER consistently improves readability and overall quality, while RAG alone offers no benefit and can introduce
hallucinations from irrelevant retrieved terms. Combining RAG with NER degrades performance in few-shot settings but improves readability when fine-tuned. Fine-tuned BioBART with NER achieves the best overall performance, highlighting entity-aware extraction as the primary driver of improved patient-friendly summaries.
Read the full abstract ▾
Computation and Language
PolERo: Studying Political Evasion in Romanian
Gabriel Stefan and Sergiu Nisioi
Political evasion refers to responses that engage with a question while withholding the requested information. Recent NLP work frames political evasion as a classification task using a two-level taxonomy of response clarity and fine-grained evasion strategies. Existing work on response clarity and evasion classification is limited to English, leaving open whether the taxonomy and model behavior transfer across
languages and political contexts. We introduce PolERo, a
dataset of 3,574 human-annotated question-answer pairs extracted from official transcripts of five Romanian presidents. We evaluate multiple classification approaches on both
datasets under matched conditions, including TF-IDF baselines, fine-tuned encoder models, a proposed sliding-window encoder, and zero/few-shot LLM prompting. We study cross-lingual transfer through joint bilingual training and machine-translation-based data augmentation. Our results indicate that fine-tuned encoders are competitive, cross-lingual transfer is asymmetric, and ambivalent evasion categories involving pragmatic cues remain the main challenge across all model families.
Read the full abstract ▾
Computation and Language
MultiGhostBench: A Multilingual Benchmark for Long-Form LLM-Generated Text Attribution under Distribution Shifts
Matteo Greco, Anudeex Shetty, Andrea Tagarelli et al. (+1 more)
While existing work on LLM authorship attribution (AA) has made progress, available
benchmarks remain limited, often focusing on English, controlled settings, or relatively outdated models, with the few
multilingual studies considering only relatively short texts. We introduce MultiGhostBench, a
multilingual benchmark comprising 928 books generated by five recent LLMs across six
languages and three scripts, with an average length of approximately 59K words per book. The
benchmark supports evaluation under domain, author, and
language shifts. Evaluation of representative AA methods shows that no single method consistently performs best across settings, and performance generally degrades under distribution shifts.
Transformer-based detectors can retain generator-related information across
languages, although transfer effectiveness varies by
language pair, whereas statistical and fingerprint-based detectors are more
language-dependent. We envision MultiGhostBench as a valuable resource for the development and evaluation of robust AA methods. The
dataset and code can be found at https://github.com/GrecoMT/MultiGhostBench.
Read the full abstract ▾
All authors Matteo Greco, Anudeex Shetty, Andrea Tagarelli, Jey Han Lau
Computation and Language
NE-R1: Enhancing Named Entity Recognition Model via Reinforcement Learning
Meixuan Chen, Hehan Li, Ruizhi Zhao et al. (+8 more)
Named Entity Recognition (NER) has achieved substantial progress since the advent of large
language models (LLMs). Nevertheless, the recognition of long-tail and domain-specific entities remains challenging due to the deficiency in parametric knowledge.
Retrieval-augmented generation (RAG) offers a promising remedy by injecting external knowledge, but it also introduces noise and unnecessary cost when dealing with familiar cases. In this paper, we propose NE-R1, a novel framework for adaptive
retrieval-augmented NER. We design a "
retrieval-on-demand" mechanism for NER. Then we integrate it into models by a two-stage training method: (1) multi-task instruction tuning initialization; (2) end-to-end RL optimization with CoT. To achieve reasonable selection between parameterized and external knowledge, we design a multi-dimensional reward considering both accuracy and
retrieval benefit. NE-R1 achieves state-of-the-art performance on various
benchmarks, with an average F1 score gain of 2.52% in in-domain evaluation and 1.18% in zero-shot cross-domain evaluation.
Read the full abstract ▾
All authors Meixuan Chen, Hehan Li, Ruizhi Zhao, Xin Lu, peizhi xu, Liwei Qian, LI Meifang, shuanglong li, Hanmeng Liu, Xin Pei, Yanbiao Ma
Computation and Language · Cross-listed from cs.SD
SonicCaps: Large-Scale Diverse and Fine-Grained Captioning for Improved Audio-Retrieval
Zineb Lahrichi, Marc Ferras, Gaël Richard et al. (+1 more)
Recent advances in audio-
language modeling have been driven by large-scale audio captioning
datasets. However, existing
datasets remain limited by low semantic diversity, generic descriptions lacking acoustic details, and one-to-one audio-caption mappings that poorly reflect the inherent ambiguity of auditory perception. We introduce SonicCaps, a large-scale audio captioning
dataset comprising ~15M captions paired with ~700k audio clips, generated using a multi-modal large
language model (Qwen3-Omni) conditioned on both audio and text. To explicitly promote diversity, we generate around 24 captions per audio via structured
prompt engineering and few- shot generation, spanning main descriptions, rephrased variants (verbosity, style) and semantic tags. Human evaluation shows that SonicCaps is rated significantly higher than existing captioning
datasets, with fine-grained analyses indicating that our captions are perceived as more descriptive and precise, which strongly correlates with quality judgments. Finally, training CLAP models on SonicCaps with a multi-caption sampling strategy consistently improves audio
retrieval and zero-shot classification, with stronger generalization across public and commercial
benchmarks. We release both SonicCaps and two specialized CLAP models on hugging face: https://huggingface.co/
datasets/Zineb/SonicCaps.
Read the full abstract ▾
All authors Zineb Lahrichi, Marc Ferras, Gaël Richard, Geoffroy Peeters
Computation and Language · Cross-listed from cs.AI
SALA: Semantic-Aware Logical Alignment for Complex Reasoning in In-Context Learning
Zhao Ji, Wenqing Chen, Zhixuan Chu et al. (+4 more)
Effective in-context learning (ICL) for complex
reasoning relies on selecting the right demonstrations. Traditional
retrieval methods based on surface similarity fail to capture the underlying problem-solving logic. Recent logic-based methods address this by matching predefined
reasoning steps, but the rigid rules and exact-match criteria is improper to handle flexible or diverse
reasoning processes. To address the problem, we propose SALA, a Semantic-Aware Logical
Alignment framework. Instead of relying on a fixed inventory, SALA automatically learns task-specific
reasoning operations. It then embeds these operations into a continuous semantic space and uses dynamic time warping (DTW) to align the
reasoning sequences. This approach allows for soft, flexible matching of
reasoning logic while remaining highly interpretable. Experiments across four
reasoning benchmarks and three LLMs demonstrate that SALA outperforms existing demonstration selection methods. Further analysis confirms the roles of the operation induction and the logical semantic
alignment.
Read the full abstract ▾
All authors Zhao Ji, Wenqing Chen, Zhixuan Chu, Jianxing Yu, Jingping Liu, Shanhe Zhao, Zibin Zheng
Computation and Language · Cross-listed from cs.IR
Counter-GEO-Bench: Evaluating Defenses Against Information-Distorting Generative Engine Optimization
Bing Zheng, Zongyao Zhao and Wenming Yang
Generative engine optimization (GEO) enables content producers to increase the visibility of their web pages in generative search engines, but the same techniques can deliver targeted misinformation when adversaries publish ordinary-looking GEO-optimized documents that victim large
language models (LLMs) retrieve and synthesize into distorted answers. No existing
benchmark evaluates defenses against this threat under controlled conditions. Therefore, we present Counter-GEO-Bench, a defense
benchmark that pairs 247 human-verified, quality-gated queries with information-preserving and information-distorting GEO rewrites, and evaluates defenses on attack success rate (ASR), false positive rate, and answer quality across three victim LLMs. Under Counter-GEO-Bench, three off-the-shelf defenses (Granite Guardian, Llama Guard 3, and NeMo Self-Check Fact-Checking) reduce ASR by at most 5.7% relative, while Granite Guardian's reduction is not statistically significant. Safety-taxonomy guardrails target policy violations, while GEO misinformation passes through them as fluent informational content. To this end, a lightweight
benchmark baseline, C-GEO Guard, is proposed, reducing ASR by 47.6% relative with near-zero utility loss, which proves threat tractable.
Read the full abstract ▾
Computation and Language
DiffIE: Diffusion-based Open Information Extraction
Konstantin Fedorov and Valentin Malykh
A single sentence often expresses multiple valid relational triplets, which makes Open Information Extraction (OpenIE) fundamentally a multi-output task. Existing neural systems handle this by autoregressive generation, which is flexible but slow and prone to redundancy, or by fixed-slot prediction, which is efficient but couples the extraction budget to training. We introduce DIFFIE which instead treats the stochasticity of conditional discrete
diffusion as the extraction mechanism itself: independent reverse-
diffusion trajectories over per-
token role tags produce a pool of candidate triplets, which are clustered under lenient matching and ranked to form the output. Both the pool size and the number of returned extractions are inference-time choices, decoupling the extraction budget from training and exposing test-time compute as a tunable axis. DIFFIE achieves the new state of the art in CaRB (1-1) both F1 and AUC, and outperforms the strongest rule-based system (ClausIE) in BenchIE; it also remains competitive in standard CaRB and WiRe57 evaluations, giving the best average score among systems that report all four
benchmarks. Ablations show that uniform discrete
diffusion outperforms absorbing state
diffusion in our setting, and that a matched non-
diffusion stochastic tagger does not reproduce its gains. Our results indicate that
diffusion stochasticity is an effective mechanism for structured prediction tasks with multiple valid outputs.
Read the full abstract ▾
Computation and Language
Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization
Bizhe Bai, Jiakang Yuan, Hongming Wu et al. (+8 more)
GUI
agents increasingly operate across websites, mobile apps, and desktop environments, yet the field still reports progress primarily through task success. We argue that practical deployment depends equally on efficiency: how much context, computation, action budget, and runtime overhead an
agent consumes while succeeding. This survey studies efficient GUI
agents through an end-to-end systems lens that preserves the current technical axes of observation efficiency, context and
memory efficiency, action efficiency, and planner-side/system efficiency. For each subsection, we expand the seed literature through targeted search plus backward and forward citation chaining, then synthesize the dominant mechanisms, reported efficiency signals, and new overheads they introduce. Across the literature, recent progress converges on a small set of recurring ideas: selective reading instead of full-context ingestion, global-to-local visual allocation, recoverable
memory rather than raw history replay, verification-aware control, and hybrid runtimes that can switch between GUI and non-GUI execution. We conclude by identifying the main open problems, including honest accounting of verifier cost, cross-
benchmark comparability, and co-design of observation,
memory, and execution layers under real latency and
privacy constraints.
Read the full abstract ▾
All authors Bizhe Bai, Jiakang Yuan, Hongming Wu, Xinyue Wang, Jie Ren, Siyao Chen, Yuchen Ya, Fan Bai, Pai Peng, Huafeng Qin, Tao Chen