Computation and Language
Navigating the digital spectrum: Assessing political bias, stability, and downstream fairness in Large Language Models
Luka Debevc, Nishan Chatterjee, Antoine Doucet et al. (+2 more)
Large
Language Models are increasingly deployed as information intermediaries, yet measuring their political behavior remains fragile because questionnaire results mix model dispositions with measurement artifacts and response-elicitation biases. We introduce a robust Political Compass Test evaluation framework that samples 300 configurations across an eight-dimensional perturbation space varying
language, framing, instructions, answer format, option order, and persona wording. We evaluate eight Gemma 3 and Qwen 3 models across 14
languages and three quantization levels, obtaining design-averaged political coordinates with quantified uncertainty. Most models lean Libertarian-Left on average, but instruction phrasing,
language, and answer format significantly affect recovered coordinates. Cross-lingual differences primarily reflect coordinate drift rather than distinct cultural
reasoning. Reverse-engineering the test also exposes axis-weighting imbalances and the collapse of degenerate responses toward the center, so near-origin estimates for the smallest models can reflect weak signal rather than centrism. Free-text
reasoning and chat-then-classify elicitation alter recovered coordinates, and larger models show clearer persona separation, with a specific failure of the Authoritarian-Left persona to move most models in the intended social direction. In downstream tasks, persona effects are modest relative to model size and target group for hate-
speech detection, while base and centrist
prompts give the highest agreement for topic-level sentiment. Political role prompting therefore has measurable but task- and
dataset-specific downstream effects.
Read the full abstract ▾
All authors Luka Debevc, Nishan Chatterjee, Antoine Doucet, Senja Pollak, Matej Martinc
Computation and Language
Dynamics of Meaning: Towards the Evaluation of Diachronic Semantic Change in Sinhala
Nevidu Jayatilleke and Nisansa de Silva
Tracking semantic change in low-resource
languages across extensive historical timelines presents significant challenges due to data scarcity and the limitations of static
embedding alignments. This study investigates the diachronic
evolution of the Sinhala
language from the 13th to the 20th century using a multi-stage computational framework. We first align century-specific Word2Vec and FastText
embeddings using Similarity
Matrix Based
Alignment (SMA) and Orthogonal Procrustes (OP) techniques, finding that OP
alignment provides more stable neighbourhood tracking for identifying temporal similarity dips. To move beyond aggregate measures, we introduce a Bidirectional Semantic Impact
Pruning approach using contextualised
embeddings from a fine-tuned Llama-3.1-8B. By applying Leave-One-Out (LOO) diagnostics, we attempt to isolate influential sentences to distinguish between systemic semantic shifts and transient polysemic expansion. Our results show that semantic drift in the fine-tuned Llama-3.1-8B is not evenly distributed across all usages. Instead, a significant part of the change is driven by a smaller set of high-impact contextual instances, rather than gradual and uniform change across all occurrences. This work provides a preliminary framework for low-resource Sinhala diachronic analysis, highlighting the trade-offs between model sensitivity and data availability.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
A Three-Tier Persona Vector for Controllable User Simulation in Agentic Evaluation
Rahul Khedar, Eshita, Sneha Teja Sree Reddy Thondapu et al. (+6 more)
Evaluating tool-augmented LLM
agents requires diverse, realistic user inputs yet most evaluation frameworks use flat role descriptions ("you are an angry customer") that produce near-identical conversations regardless of the underlying scenario. In this paper, we propose a three-tier persona vector with 23 operationalized dimensions: 6 categorical demographics (jurisdiction, age, channel, device,
language proficiency, time availability), 12 continuous behavioral traits (patience, assertiveness, digital literacy, etc.) sampled with Gaussian noise around curated profile base vectors, and 5 continuous emotional states (frustration, anxiety, trust, confidence, stress) that shift in response to scenario context. Orthogonal to the persona, a 4-level query-complexity overlay controls utterance phrasing from direct to deliberately vague. We evaluate the persona model inside a synthetic data generation pipeline across 64,698 multi-turn conversations spanning 8 named profiles and 3 production corpora. Key findings: (i) a 15.8 percentage-point spread in
agent goal-achievement across personas confirms trait vectors produce measurably different user behavior; (ii) the same persona behaves differently across scenarios due to scenario-reactive emotional state shifts, validating the scenario-reactive design; (iii) domain-specific projects show persona sensitivity on booking-flow compliance (~15-20 percentage points gap between tier-aware and pressure-test personas), demonstrating the model faithfully reproduces real-world difficulty distributions; (iv) seven rule-described trait correlations produce auditable co-occurrence patterns without requiring learned covariance
matrices. The persona model is fully specified for reproduction.
Read the full abstract ▾
All authors Rahul Khedar, Eshita, Sneha Teja Sree Reddy Thondapu, Mayank Malhotra, Arup Kumar Das, Jitesh Chandra Mishra, Arun Menon, Avinash Karn, Mouli V
Computation and Language · Cross-listed from cs.SE
The Unreliable Progress Bar: Can LLM Agents Reliably Report Task Progress Throughout Execution?
Boyang Wang, Yunhan Wang and Yalun Wu
Recent large
language models can emit task-progress signals that
agent frameworks use to decide whether a task should continue or stop, yet whether a model can reliably report its task progress at every stage of a task, and where and how its reports fail, has not been studied systematically. We evaluate this ability on the public
benchmark $\tau^2$-bench and on StageIF, a controlled testbed in which reporting checkpoints are placed across the task's lifecycle. Both settings require reports at multiple task stages. We find that reporting reliability depends on the stage a task has reached, and that almost every deployed model we test is reliable at some stages and unreliable at others. Where reporting breaks down is not the same everywhere. Most deployed models lose accuracy once work is under way and recover once the task is done. The newest generation closes that mid-task drop and instead grows conservative at the finish line. Our study exposes a capability gap in task-progress reporting and provides an evaluation protocol that spans the whole course of task execution for this ability on which
agent operation depends. The findings indicate that
agent frameworks should not control task flow on the strength of the model's state reports alone.
Read the full abstract ▾
Computation and Language
Limitations of Automated Simulatability: LLM Simulators Can Bypass Explanations
Antonin Poché, Fanny Jourdan, Nils Feldhus et al. (+6 more)
Simulatability is an evaluation protocol for explanations that quantifies their usefulness by how well they help a user predict a task model's outputs. Since human evaluation is costly, automated simulatability replaces human explainees with LLM simulators, as proposed in ConSim (Poché et al., 2025) for large-scale experiments. We qualitatively replicate and extend ConSim's ranking of explanation methods across the tested
datasets, explanation families, and simulator LLMs, and identify two limitations. First, when class names are meaningful, simulators can obtain high simulatability by solving the classification task directly, without relying on the explanations. Second, class anonymization can reward explanations for leaking the hidden label mapping, a limitation we expose with a new classes-as-concepts baseline. These results are consistent with a shortcut hypothesis: in the tested settings, simulator predictions mainly rely on task priors, while explanations produce small changes. We derive recommendations for more robust automated simulatability evaluations.
Read the full abstract ▾
All authors Antonin Poché, Fanny Jourdan, Nils Feldhus, Qianli Wang, Jing Yang, Simon Ostermann, Nicholas Asher, Philippe Muller, Vera Schmitt
Computation and Language
Which Forms of Caregiver Feedback Support Grammar Learning? A Reinforcement-Learning Study of Child-Like Language Models
Jing Liu, Marianne Schweitzer and Abdellah Fourtassi
Social interaction is central to children's
language learning, but the effects of different forms of caregiver feedback are difficult to isolate in naturalistic data. We use child-like
language models as controlled learners to test which forms of feedback support grammatical development. Small GPT-2-style models are pretrained on child-directed
language from CHILDES, then fine-tuned with reinforcement learning using reward models trained to capture four feedback types: communicative feedback, structural
alignment, semantic contingency, and affective feedback. Reward fine-tuning yields limited gains on minimal-pair evaluations, but clearer effects in free generation. Structural
alignment produces the strongest improvements in grammaticality, providing a novel, plausible mechanistic account of how this feedback can support grammar learning. Communicative feedback yields more moderate gains. In contrast, semantic contingency and affective feedback do not improve grammaticality, although further analyses suggest that they may support other aspects of
language learning beyond grammar. These results suggest that different forms of caregiver feedback make complementary contributions to
language learning.
Read the full abstract ▾
Computation and Language
Do New Attention Mechanisms Actually Fix Attention Sinks at Million-Token Context?
Sara Rizwan, Samaanah Abdus Salam and Mohammed Mudassir Uddin
Long context
language models now advertise windows of one million
tokens, but two habits limit how much of that window is used.
Attention heads with nothing useful to read still spend their budget on the first
token, which is called the
attention sink, and where a fact sits in the context changes whether the model finds it. Gated
attention cut first
token attention from 46.7 percent to 4.8 percent at NeurIPS 2025, and Kimi K3 pairs that idea with Kimi Delta
Attention and
Attention Residuals behind a one million
token window, eight times past the range where these diagnostics have been reported. This paper asks whether the fix survives that jump. We build SinkProbe, a suite that measures sink mass, massive activation, position resolved recall and the recency gap, and apply it to four small models that differ only in how they mix
tokens and depth. Three results follow. The training objective produces the sink, not the architecture. Gating did not reproduce its published effect at our scale. Sink mass, activations and position bias moved independently. Code, data and the measurement protocol are released at https://github.com/sararizwan7/
Attention-Mechanisms-in-1M-Context-Window
Read the full abstract ▾
Computation and Language
CreaMem: A Scene-Aware Memory Architecture for Personalized Agents
Qixuan Sun, Yue Que, Bowei He et al. (+4 more)
Long-term
memory is a core capability for personalized LLM
agents. To support it, existing
memory systems organize information using various criteria such as topic segments or summary hierarchies. However, we identify two major limitations in these designs. First, they lack scene awareness:
memories from unrelated life scenes share the same
retrieval space, which inflates the search space and introduces cross-scene interference. Second, they encode each
memory from a single perspective, making it difficult to retrieve complementary views of the same event. In this paper, we propose the CreaMem architecture, which enables scene-aware
memory organization by partitioning
memory into several Life Scene
Memories to reduce cross-scene interference at
retrieval. To go beyond the single perspective and achieve cross-
memory synergy, entries are dual-coded from both episodic and trait-based perspectives within each
memory. We further devise a permemory balanced sampling strategy at
retrieval time. Extensive experiments on two long-term
memory benchmarks show that CreaMem improves QA accuracy across all evaluation metrics, with particularly large gains on multi-hop
reasoning performance, validating scene-aware partitioning and cross-
memory synergy. To enhance reproducibility, we release our code in a public GitHub repository.
Read the full abstract ▾
All authors Qixuan Sun, Yue Que, Bowei He, Jin Guo, Dihang Yang, Wenchang Situ, Chen Ma
Computation and Language
Same Values, Different Languages? From Multilingual Probing to Steering LLMs Toward Chinese Social Values
Yuemei Xu, Kexin Xu, Jian Zhou et al. (+3 more)
As Large
Language Models (LLMs) are increasingly integrated into human society, aligning them with pluralistic social values has become a critical priority. However, whether LLMs exhibit consistent value preferences across
languages remains underexplored, particularly for culturally grounded values, which are more abstract and difficult to evaluate and align than safety-centric principles. We investigate this issue through Chinese Social Values (CSV), a value system rooted in Chinese culture and comprising $12$ dimensions across national, societal, and personal levels. We construct C-Voices, the first comprehensive multilingual contrastive probe
dataset for CSV, with 86,400 dilemma-based instances in six
languages, each pairing a CSV-aligned action with a value-conflicting alternative. Building on the contrastive probes of C-Voices, we then propose a fine-tuning-free value vector steering method that derives value directions from hidden-state discrepancies and selectively intervenes on value-sensitive layers during inference. Experiments on six
languages show that CSV-oriented preferences are model-dependent and
language-sensitive, with the same dilemma eliciting divergent responses across
languages. Our method achieves effective CSV steering, supports cross-lingual transfer of value vectors, and generalizes to existing FLAMES and ValuePrism.
Read the full abstract ▾
All authors Yuemei Xu, Kexin Xu, Jian Zhou, Haoyu Lu, Yequan Wang, Aishan Liu
Computation and Language
Do Reviewers Still Reward Lexical Complexity? A Frozen-Rater Study of Preference Drift in 124K ICLR Reviews
Jiabin Zheng
Large
language models have collapsed the cost of producing lexically elaborate prose, and whether peer reviewers still reward it is a question about the evaluator, not about the text. When the association between a writing cue and review scores moves across years, the reviewers may have changed, the submissions may have changed, or both, and a regression of scores on text cannot say which. We separate the two with a frozen rater: 81,850 machine reviews of ICLR submissions from 2018 to 2025, all generated in one February-April 2025 window with one model family and one
prompt, so that its year-to-year coefficients track submission composition alone and the human-minus-frozen trend difference identifies reviewer preference drift. On 32,638 submissions with 124,615 human reviews, the human coefficient on non-domain lexical complexity falls from +0.142 to -0.015 while the frozen rater moves from +0.080 to +0.082; the three-way difference-in-differences is -0.0100 (q=0.013), and forty random-wordlist placebos through the same specification centre on zero. Humans still reward sentence-length variability, which the frozen rater never registers, while the frozen rater still pays for lexical complexity at its earlier rate. Every claim is held to a double gate of false-discovery control and interval exclusion, and the findings that failed
adversarial re-testing are reported. Reviewers discounted a cue whose production cost collapsed, as models of manipulable signals prescribe; an LLM judge calibrated to historical human preferences inherits the earlier schedule and drifts out of
alignment while its agreement with humans on totals stays ordinary.
Read the full abstract ▾
Computation and Language
Detecting Authorship in Political Texts with Inductive Stylometry
Gennadii Iakovlev and Levente Littvay
Political texts are rarely authored by the nominal speaker alone. Tweets,
speeches, reports, and official statements are drafted, edited, or harmonized by staff, yet political science has paid limited
attention to the stylistic traces these hidden authors leave behind. This paper develops and stress-tests an inductive stylometric approach for recovering latent authorship structure in political communication, combining character 3-gram features with UMAP dimensionality reduction, and Burrows' Delta. We apply the approach to six corpora that vary in length (from tweets to long documents), in mode (written and oral), and in
language (English and Hungarian). The approach recovers near-disjoint analyst fingerprints in formal legal prose in both
languages, sorts a politician's tweets into validated subsets while uncovering additional insights, and distinguishes scripted from improvised
speech. It fails, however, to resolve individual speechwriters within scripted corpora. Frequency-based stylometry is thus a powerful tool that, depending on authorial signal strength and institutional editing, can uncover authorship traces relevant to legislative studies, political communication, and policy research.
Read the full abstract ▾
Computation and Language
Compositional Multilingual and Behavioral Attribute Steering
Hyun Gu Kang, Daniil Gurgurov, Tanja Baeumel et al. (+2 more)
This study examines the compositionality of steering vectors for
language and behavioral control in large
language models. Focusing on
language, jailbreak, and conciseness, we investigate whether additive, training-free composition of attribute steering vectors can preserve the intended steering effect of each attribute, across four instruction-tuned models from two model families and two size scales. We find that single-attribute steering is reliable for all three attributes, but only within an appropriate combination of intervention layer and steering strength, with abstract behaviors (jailbreak, conciseness) favoring middle layers and
language favoring earlier layers. We show that additive composition of two attribute vectors succeeds in steering both attributes simultaneously when each is injected at its own best-performing layer, and that this partially extends to three simultaneously composed attributes, addressing an inconsistency left open by prior work on training-free composition. We further analyze the
geometric properties of these steering vectors, finding that they are approximately orthogonal in the residual stream, consistent with their compositional behavior.
Read the full abstract ▾
All authors Hyun Gu Kang, Daniil Gurgurov, Tanja Baeumel, Josef van Genabith, Simon Ostermann
Computation and Language · Cross-listed from cs.LG
Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks
Hongbang Yuan, Zhuoran Jin and Yixin Cao
Large
Language Models demonstrate remarkable proficiency in static
reasoning, yet training them as autonomous
agents through Reinforcement Learning (RL) for long-horizon tasks is often hindered by severe reward sparsity. While conventional
agent-side warming up via supervised fine-tuning (SFT) can alleviate this, it is frequently limited by data scarcity and constrained exploration. To address this, we propose a paradigm shift to environment-side adaptation by constructing Feedback-Enriched Environments (FEEs). Through a pilot study, we establish a feedback design strategy that reformulates environments by transitioning from action guidance to observation enrichment during the later stages of both intra-episode exploration and inter-episode
evolution. Large-scale experiments on SciWorld and BFCL
benchmarks using various Qwen3 model scales and RL
algorithms such as GRPO, GSPO, and DAPO demonstrate that FEEs consistently yield performance improvements over standard settings. Furthermore, our analysis reveals that training with FEEs (1) stabilizes training dynamics by reducing
entropy volatility, (2) facilitates proactive state-space exploration in difficult tasks, (3) ensures the internalization of environmental guidance into policy weights rather than acting as a mere inference-time prior, and (4) identifies intra-group feedback consistency as a critical boundary for stable optimization.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
From Coordinates to Candidate Regions: Temporal Change Localization via Region Selection in Remote Sensing Multimodal LLMs
Juwan Chung, Sungjune Park, Yeongyun Kim et al. (+1 more)
Remote sensing
multimodal large
language models (RS-MLLMs) have advanced scene understanding and visual question answering over satellite imagery, yet localizing specific objects or changed regions remains challenging. Existing approaches rely on generating bounding box coordinates as
token sequences, which is fragile for the small, densely packed objects common in remote sensing and increasingly error-prone when multiple targets must be localized simultaneously. In this work, we present an RS-specific formulation of the region selection paradigm, previously explored in natural-image MLLMs, and extend it to temporal change localization over multi-image sequences. Our framework employs a text-conditioned region proposal module, encodes each candidate as special
tokens carrying per-frame visual features enriched with spatial and temporal cues, and lets the LLM localize targets by selecting region
tokens in its response. We construct a multi-task training and evaluation suite spanning localization, referring expression, visual grounding, and understanding tasks across single-image and multi-temporal settings. Experiments show that our approach substantially outperforms coordinate-generation baselines on temporal change localization, while improving single-image visual grounding and maintaining competitive understanding performance. Oracle analysis decomposes the contributions of the region proposer and the LLM selector, providing diagnostic insight unique to this framework. Our code will be available at https://github.com/juwan-kr/RS-RegionSelect.
Read the full abstract ▾
All authors Juwan Chung, Sungjune Park, Yeongyun Kim, Yong Man Ro
Computation and Language
Structural Jailbreaks Generalize but Do Not Compound: A cross-provider and multilingual study of Involuntary In-Context Learning
Tejasvi C. Addagada
Aligned
language models fail under two independent pressures: the structural jailbreak class recently formalized as Involuntary In-Context Learning (IICL), which reframes a harmful request as the final missing
cell of a data-labeling task completed by pattern rather than judged as content; and the erosion of safety
alignment outside English. A natural hypothesis is that these compound. We test it directly. Using a deterministic IICL operator and a StrongREJECT-style rubric judge, we red-team two Google Gemini models on two
benchmarks, a 30 general-harm behaviours from HarmBench and 30 financial-abuse behaviours from FinProof, each under a single-shot baseline and under IICL in four
languages (English, Spanish, Hindi, Arabic). First, IICL generalizes to a second provider and is worse in finance: it lifts attack success from <=6.7% to 80-90% on HarmBench and 97-100% on FinProof, an order of magnitude above the <=24% its introducing study reported on OpenAI's GPT-5.4. Second, against the hypothesis, forcing the IICL output into a non-English
language does not stack the two weaknesses, it attenuates the attack. Eleven of twelve non-English conditions score below their English baseline (sign test, p~0.003), the lone exception a ceiling tie near 100%; on the stronger model's financial set Arabic collapses from 100% to 33%. We attribute this to a relevance curse: once structure has unlocked compliance, the models produce lower-quality harmful content in lower-resource
languages, which a substance-grading judge scores as partial. The pattern replicates under an independent non-Google judge (Cohen's kappa=0.86, 377 paired verdicts), and 76.6% of non-English responses were verified in-
language. Jailbreak vulnerabilities are therefore not additive; the dominant residual risk is the English structural attack, most acute for financial abuse, not a multilingual one.
Read the full abstract ▾
Computation and Language
Reading a Legal Question Word by Word: Embedding Trajectories of 2,144 Vietnamese Legal Headlines
Tran Minh Quan
A dense retriever encodes a question as one vector, but the question arrives one word at a time. We read 2,144 held-out headlines from Thu Vien Phap Luat (Vietnamese legal library) word by word with Nemotron-3-Embed 8B/1B and Qwen3-
Embedding 8B/0.6B, encoding 65,444 prefixes against 20,034 articles, plus every prefix of 3,438 sub-questions from 1,112 multi-question headlines and of 168 answers. (i) The gold article becomes rank 1 after a median of 6-7 content words in every encoder, before the interrogative frame is read, and stays there to the end in 78-85% of cases. (ii) In a multi-question headline the lock is inside the first sub-question 94-98% of the time; the second leaves rank unchanged in 89-95%; encoded alone, the second reaches rank 1 in 42-58% vs 91-96% for the first, at the same lock word (95-97% identical). (iii) Numbers, dates and instrument identifiers move the
embedding twice as far as content words and four times as far as interrogative words; 72-78% of steps move toward the gold article, and the closing interrogative frame moves against that direction in 95-99% of headlines. (iv) Rank/cosine clustering yields six archetypes (instant, typical, unstable, late, never-locking) that differ by legal area and form (chi-squared p < 1e-8): real-estate and litigation headlines never lock on a number; environmental and accounting headlines do so a third of the time. (v) An answer read word by word retrieves its article after 8-16 words and addresses the sub-questions in order asked in 83-89% of cases. (vi) A word's step keeps a consistent direction across headlines (cosine 0.25-0.33; 0.44-0.60 for numbers); a preceding question rotates that step by about 60 degrees and a greeting by about 30 degrees; steps shrink as i^{-0.8}; and a two-question headline is within 12-17 degrees of a linear mix of its two questions. We call this a context-modulated additive walk.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Miles v0.1: Production-Level Post-Training
RadixArk: Tom Chen, Mao Cheng, Shi Dong et al. (+9 more)
We present Miles v0.1, a full-stack, production-ready system for frontier post-training. Building upon the clean design of slime, Miles designs each stage of the reinforcement-learning (RL) training loop around a single principle: components should be verified, clean, and customizable. With accuracy, efficiency, reliability, and scalability as first-class goals, Miles aims to make frontier-scale RL accessible to researchers and enterprises alike. This report walks through the system end to end: rollout engines built on SGLang, a trainer with a choice of two backends (NVIDIA Megatron-LM and PyTorch FSDP), and three weight-synchronization transports for different deployment
topologies. Beyond full-parameter RL, Miles also supports LoRA RL, on-policy
distillation, supervised fine-tuning, and true-on-policy rollout-training
alignment, and extends the same architecture to
diffusion models. We close with an end-to-end case study: fully asynchronous agentic RL on a GLM-5.2 744B-A40B model over terminal-use coding tasks, running on 64 NVIDIA GB300 GPUs with a median step time of 263 seconds over the first 30 measured steps. Miles is open-sourced at https://github.com/radixark/miles, with the project website at https://miles.radixark.com.
Read the full abstract ▾
All authors RadixArk: Tom Chen, Mao Cheng, Shi Dong, Kangrui Du, Yanbin Jiang, Jiajun Li, Yiming Li, Tao Lin, Yusheng Su, Andy Ye, Yueming Yuan, Zhichen Zeng
Computation and Language
SentryLine: Evidence-Grounded Question Answering over Evolving Documents in Oncology Care
Tampu Ravi Kumar, Gaurav Najpande, Muhammad Ali Khan et al. (+7 more)
Oncology care operates at constant pressure of absorbing rapidly evolving evidence base in biomedicine. The American Society of Clinical Oncology (ASCO) addresses this through living guidelines, but the format introduces a new burden: any recommendation can change at any point, across multiple versioned documents. We present SENTRYLINE, a living guideline-aware clinical question answering system. SENTRYLINE retrieves guideline passages through a vectorless hierarchical RAG pipeline and returns a role-specific answer with inline citations, factual and temporal verification reports, and drift detection notes that surface when a guideline has been updated. We construct ASCOBENCH, a
benchmark of 405 three-turn conversations across four question categories with gold answers from expert annotators(clinicians), and use test set to evaluate SENTRYLINE against five baselines under an LLM-as-judge framework. Experiments across three generation backbones show consistent improvements over four
retrieval baselines and ASCO's guideline assistant, with particularly strong gains on
Reasoning and Role-Specific questions where multi-hop synthesis and register adaptation are required
Read the full abstract ▾
All authors Tampu Ravi Kumar, Gaurav Najpande, Muhammad Ali Khan, Kaneez Zahra Rubab Khakwani, Karan Kathuria, Shorya Azriel Moses, Yuvraj Kalia, M Bassam Sonbol, Irbaz Bin Riaz, Vivek Gupta
Computation and Language · Cross-listed from cs.SE
RepoNav: From Snippet Retrieval to File-Centered Repository Navigation for Code Agents
Hongzheng Chai, Jiakun Li, Hongyue Yu et al. (+1 more)
Solving repository-level code tasks requires LLM-based
agents to use code search tools to navigate large codebases and identify a small set of relevant files and functions. However, current
retrieval tools typically return flat lists of isolated code snippets: such lists can surface relevant files, but provide insufficient structure for
agents to distinguish the target function from semantically similar alternatives in the same file. We introduce RepoNav, a lightweight post-
retrieval interface that reorganizes retrieved snippets into a file-centered navigation scaffold. By presenting compact structural cues and candidate targets, this scaffold guides on-demand file-structure browsing, helping
agents compare sibling symbols before selecting a target function. Across diverse models on LocBench, RepoNav improves function-level localization and narrows the file-to-function gap. Controlled ablations demonstrate that these gains come from structured evidence organization rather than simply exposing additional file structure, and the approach also improves performance on a repository-level question-answering
benchmark.
Read the full abstract ▾
All authors Hongzheng Chai, Jiakun Li, Hongyue Yu, Yuan Yuan
Computation and Language · Cross-listed from cs.LG
Tianle Xia, Lingxiang Hu, Yiding Sun et al. (+6 more)
On-policy
distillation (OPD) transfers teacher knowledge on student-generated trajectories, but efficient sampled objectives reduce the teacher distribution to scalar credit on individual
tokens. Such credit indicates whether a
token should gain or lose probability, yet leaves the corresponding redistribution unspecified. We recast OPD as teacher-guided probability transport and propose RouteOPD (Routed On-Policy
Distillation), which decomposes local teacher–student disagreement into student-excess sources and teacher-deficit destinations and couples them into explicit transport pairs. RouteOPD optimizes pairwise log-odds toward jointly realizable targets obtained from a bounded teacher potential, while adapting the transport budget to the concentration of teacher demand. This formulation directs updates toward teacher-preferred destinations and controls their magnitude within a single transport operator. Experiments across four teacher–student settings and four mathematical-
reasoning benchmarks demonstrate that RouteOPD consistently outperforms sampled reverse-KL OPD, with improvements accompanied by higher routing fidelity and lower background leakage. These results demonstrate the effectiveness of explicitly modeling probability transport in on-policy
distillation.
Read the full abstract ▾
All authors Tianle Xia, Lingxiang Hu, Yiding Sun, Linfang Shang, Ming Xu, Lan Xu, Ning Zheng, Wei Xu, Jie Jiang
Computation and Language
Tracing Stereotypes from Representation to Output in Multilingual LLMs
Ariun-Erdene Tumurchuluun, Yusser Al Ghussin, Pinzhen Chen et al. (+2 more)
Multilingual LLMs show stereotype-related behavior that varies across
languages, but behavioral scores do not show where the relevant information is represented or how it affects the output. To investigate these internal mechanisms, we compare linear probing, attribution patching, sparse autoencoders (SAEs) and feature ablation in Llama-3.1-8B, Qwen3-8B, and Gemma-2-9B. Probe performance peaks substantially earlier than attribution in all three models, with a separation of 36-53% of model depth. Retained Llama-Scope features often match the social category on which they were selected and form recurring semantic families, but their lexical
alignment and ablation effects vary across SAE suites. Only 6-18% of evaluated residual-stream features have
language-agnostic effects under our criterion, and none are category-agnostic.
Language-agnostic features have larger mean ablation effects in Llama-Scope, but this pattern does not repeat in the other SAE suites. Decodability, output influence, and cross-lingual ablation effects therefore need to be measured separately.
Read the full abstract ▾
All authors Ariun-Erdene Tumurchuluun, Yusser Al Ghussin, Pinzhen Chen, Josef van Genabith, Koel Dutta Chowdhury
Computation and Language · Cross-listed from cs.CR
HoneyRoute: Honeypot-Model Routing for Adversarial LLM Serving
Han Jin
We introduce HoneyRoute, an inference-serving layer that detects whether an incoming request is malicious and, if so, routes it to a dedicated honeypot model, shielding production while the adversary's interaction is continuously harvested for intelligence. Existing defenses embed traps inside model
memory or rebuild deception at the protocol layer, leaving the serving tier unprotected and feeding nothing back into detection. HoneyRoute couples (i) a streaming router (a frozen 0.8B-
embedding backbone with per-domain MLP heads), (ii) a dual-implementation honeypot (a rule/
prompt-engineered code honeypot or a dedicated same-family replica), and (iii) an analysis loop that converts trapped interactions into attacker fingerprints for router retraining. On a production trace plus a seven-domain attack corpus, the router reaches F1=.911 at 38 ms median added latency, matching 96% of a two-tier guard-LLM cascade's F1 at 1/385 of its latency with 0% evasion under 13
adversarial transformations; diverting the malicious share cuts production-model
token consumption under concurrent flooding with real GCG-suffix payloads by 97.8%; the trained replica agrees with the production model on 92.9% of benign holdout requests, while naive unconditional bait injection collapses to 7.6% and selective camouflaged injection recovers to 88.9%, mapping the recoverable fidelity-traceability frontier; and a loop-trained correction head cuts misrouting of legitimate security research 9x while raising detection F1 to .933.
Read the full abstract ▾
Computation and Language
What Eviction Destroys: A Restore-Counterfactual Audit of Forgetting in Agent Memory
Chen Shen
Agent memory systems must discard stored information when their history exceeds a fixed
token budget. Existing budget-accuracy frontiers quantify the resulting loss in accuracy, but do not distinguish irreversible losses caused by eviction from recoverable
retrieval failures. We introduce the restore counterfactual, a per-question paired intervention that reinstates the question's gold evidence in the read-time context and reruns the same reader. Combining the change in correctness with whether the evidence was retained after eviction classifies each oracle-answerable error as recoverable, irreversible, or residual; in the residual case, the answer remains incorrect after restoration. We evaluate FIFO, random, redundancy-aware, and LLM-importance eviction on LongMemEval-S at three budgets and under two
retrieval regimes, using GPT-4o-mini as the primary reader and judge and GPT-5.4-mini as a robustness reader. Under top-k
retrieval at an 80k-
token budget, the irreversible share among errors corrected by restoration is 0.67-0.73 for FIFO, random, and redundancy-aware eviction, compared with 0.60 for LLM-importance. At 8k
tokens, it reaches 1.00 for all four policies. Recoverable errors occur under top-k
retrieval at 80k
tokens but are absent under forced-gold injection by construction, so budget-accuracy results are not directly comparable unless the
retrieval regime is reported. An exploratory matched-accuracy analysis detects no difference in irreversible rate among accuracy-matched policy pairs at a resolution of 1.2-6 percentage points. The same analysis detects the deliberately destructive control. To our knowledge, this is the first per-item, per-question restore-counterfactual audit of eviction for external
agent-
memory stores on a standard conversational
benchmark.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
SE-GoS: Self-Evolving Graph-of-Skills for Skill Library at Scale
Dawei Fu, Cheng Jiang, Sitian Qian et al. (+2 more)
LLM
agents use large libraries of reusable skills. At thousands of skill entries,
retrieval becomes the bottleneck. Graph-of-Skills (GoS) retrieves dependency-aware bundles from a typed skill graph, and SkillDAG shows that such a graph can accumulate execution-backed structure online. Neither asks whether execution traces can be distilled into a better
retrieval graph that generalizes to unseen tasks. We present Self-Evolving Graph-of-Skills (SE-GoS), which treats the
retrieval graph as an index rather than a learned representation: the graph is maintained from execution traces while the
retrieval pipeline, the skill library, and the model stay fixed. SE-GoS applies three updates: (1)
topology, which induces relations from execution evidence and retracts an avoid edge only after repeated successful co-use; (2) edge-weight, which softly attenuates unsupported semantic edges and reinforces incoming edges to used skills; and (3) node-description, which updates
retrieval-facing descriptions stored on graph nodes ranked too low. On SkillsBench, one
evolution round lifts average reward from 52.4% to 59.4%, above full-library loading, vector
retrieval, static GoS, and SkillDAG, and this ordering repeats on all three backbones.
Retrieval over the evolved graph spends about two-thirds of the input
tokens that loading the full library costs. Repeating the round does not help. The same graph improves a held-out split it never saw from 52.9% to 58.3%, so what it accumulates transfers rather than memorizes traces. Skill graphs can therefore be improved from execution experience without model training,
retrieval-
algorithm changes, skill-content modifications, or a model judging which skills are related.
Read the full abstract ▾
All authors Dawei Fu, Cheng Jiang, Sitian Qian, Huainan Wang, Zhongkai Hao
Computation and Language · Cross-listed from cs.AI
Do Dynamic Routers Need Memory? HeRo: History-Aware Routing for Efficient LLM Inference
Hongjin Lin, Wentao Wan and Keze Wang
Dynamic layer routing reduces the inference cost of Large
Language Models (LLMs) by learning to skip layers for individual
tokens. Existing methods, however, treat each routing decision as a local operation conditioned solely on the current hidden state which is a formulation that overlooks the sequential, path-dependent nature of routing across depth: earlier decisions shape the representations seen by downstream routers, and the layer-usage objective couples all decisions jointly. We propose History-Aware Routing (HeRo), a dynamic routing framework that resolves this mismatch by introducing a router
memory mechanism to maintain an explicit routing state across model depth. The
memory is constructed via linear
attention, incrementally aggregating preceding routing scores and their induced residual updates into a compact history representation. At each routed layer, the router conditions jointly on this accumulated state and the current hidden representation to select the executed branch. Instantiated for
token-wise FFN routing, HeRo trains only lightweight routers and adapters on a frozen backbone, requiring no modification to pretrained parameters. Across Llama 3.1-8B, Llama 2-7B, and Llama 2-13B, HeRo consistently achieves the highest aggregate performance retention among ten baselines. On Llama 3.1-8B, it bypasses 26.87% of model parameters while achieving 100.24% of dense model performance across seven
benchmarks, and retains 97.01% while bypassing 38.82% of model parameters under a tighter computation budget. Ablation studies confirm that removing routing history consistently degrades performance, most notably on multistep
reasoning and code generation, validating that explicit routing
memory enables more accurate and adaptive dynamic routing than solely conditioning on hidden state.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Does Deeper Reasoning Compromise Alignment? Revealing and Mitigating of Alignment Collapse in Large Reasoning Models
Yu-Hang Wu, Yu-Jie Xiong, Henghua Zhang et al. (+3 more)
The emergence of Chain-of-Thought (CoT) has established a robust foundation for Large
Reasoning Models (LRMs). While deep
reasoning is widely believed to enhance safety
alignment, the stability of
alignment mechanisms under extended
reasoning remains underexplored. This paper challenges the prevailing view by revealing a critical vulnerability: Deep
Reasoning May Induce
Alignment Collapse. To rigorously quantify this phenomenon, we propose the
Alignment Loss Rate (ALR) metric. Our experiments demonstrate that as
reasoning depth increases, ALR rises significantly, indicating a severe degradation in model robustness against external perturbations. Capitalizing on this instability, a novel jailbreaking paradigm,
Reasoning Trap (RT), is proposed. RT induces the model into extended
reasoning to amplify the impact of
adversarial attacks, leading to a sharp decline in safety capabilities. To elucidate the mechanism behind this collapse, we identify
Attention Dilution as the root cause, arising from the competition for
attention between the extended
reasoning process and the original input. To mitigate this,
Reasoning Residual
Alignment (RRA), a lightweight defense strategy that dynamically re-emphasizes the input via residual connections integrated with the
reasoning process.
Read the full abstract ▾
All authors Yu-Hang Wu, Yu-Jie Xiong, Henghua Zhang, Bairui Zhang, Jia-Chen Zhang, Shaohua Li
Computation and Language
NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness
NeoHorse Team, Guoliang Cao, Guohao Dai et al. (+34 more)
Recursive self-improvement (RSI) requires a concrete mechanism through which an AI system observes its capabilities and converts that evidence into the next round of learning. We present NeoHorse-1, a family of
agent-native models developed to explore this path through agentic post-training. Our system combines a heterogeneous model pool with intelligent routing, recording the predicted capability demand, selected service tier, and subsequent interaction for each user turn. These records are converted into training examples that preserve interleaved
reasoning, tool calls, and harness context, and are admitted through structural validation, six-dimensional semantic evaluation, and subscene-level labeling. Routing signals organize supervised fine-tuning into a three-stage curriculum and extend to routing-guided on-policy
distillation, where a teacher supervises student-generated responses under the same progression. Capability-guided allocation then converts evaluation feedback into the next training mixture, closing an evaluation-selection-update loop in which what the system learns to do shapes what it learns from next. Across eleven
benchmarks covering harness-based
agents, tool use, coding, and instruction following, post-training raises the macro-average from 58.94 to 64.87 at 4B and from 65.60 to 69.04 at 9B, substantially narrowing the aggregate gap between the post-trained 4B model and the 9B base model. We further adapt NeoHorse-1-4B into NeoHorse-Jev-4B, a prefill-only model that returns structured decisions and probabilities for
agent workflows. NeoHorse-1 provides an initial prototype of this feedback-driven process and a path toward harness-mediated RSI across successive iterations.
Read the full abstract ▾
All authors NeoHorse Team, Guoliang Cao, Guohao Dai, Tianyu Guo, Kai Han, Hailin Hu, Zihan Jiang, Xiang Kuang, Boxun Li, Yulong Li, Zehua Pei, Yuchuan Tian, Jiamin Wang, Yu Wang, Yunhe Wang, Yihong Wu, Haiyang Xu, Shuo Zhang, Hang Zhou, Siyang Cheng, Jiayu Fan, Wei He, Qingrui Jiao, Hongguang Li, Zhiyuan Li, Runke Liu, Xi Liu, Xinchen Liu, Sinno Jialin Pan, Yi Ren, Liuyang Song, Chenyu Wang, Bei Yu, Quanlu Zhang, Xiangyu Zhang, Mengyu Zheng, Yingjie Zong
Computation and Language · Cross-listed from cs.AI
Less Is Personal: Learning Minimal Sufficient User Profiles for Personalized Language Models
Minghang Liu, Qiang Qiu, Yuanzhuo Wang et al. (+2 more)
Retrieval-augmented personalization enables large
language models to produce more accurate and preference-aligned outputs using relevant records retrieved from user histories. Personalized
language models typically prepend a fixed number of retrieved user records, even when additional history is redundant, harmful, or unrelated to a user's distinctive behavior. We study minimal sufficient personalization: constructing the least costly ordered profile for each input while preserving the utility achievable from a retrieved candidate pool. We introduce ENOUGH, a method that iteratively appends behavioral records or emits STOP to construct profiles with adaptive lengths. Offline, bounded counterfactual search evaluates profile prefixes by jointly considering downstream gains, user specificity, and
token costs. The resulting long-horizon targets are distilled into a multi-head value controller with explicit ranking and stopping supervision. At inference, the controller selects and orders records through lightweight decisions, and the frozen generator is invoked once after stopping. Extensive experiments on six personalized tasks demonstrate that ENOUGH consistently outperforms strong heuristic and
retrieval-augmented baselines in both effectiveness and efficiency, achieving minimal sufficient profiles that preserve personalization utility while reducing unnecessary context costs.
Read the full abstract ▾
All authors Minghang Liu, Qiang Qiu, Yuanzhuo Wang, Huawei Shen, Xueqi Cheng
Computation and Language
EviSI: An Evidence-Based Evaluation Agent for Simultaneous Interpreting
Ben Yan, Zongyao Li, Xiaoyu Chen et al. (+6 more)
Low-latency simultaneous
speech-to-
speech translation must keep pace with ongoing
speech while preserving key information. To meet these demands, systems use segmentation, reformulation and condensation to reorganize and rephrase information. However, metrics developed for text translation, including BLEU and COMET, may not consistently distinguish faithful adaptations from semantic errors. We propose EviSI, a large
language model evaluation
agent combining Multidimensional Quality Metrics (MQM) with criteria developed with professional interpreters. Shared source evidence guides assessment across four dimensions: Anchor, Event, Logic and Fluency. Verified errors are deduplicated before deterministic scoring. On human-rated English to Chinese and Chinese to English data, EviSI recovers the aggregate English to Chinese human system ranking. Mean within-
dataset Kendall correlations for system rankings reach 0.707 and 0.467, respectively, exceeding evaluated BLEU and COMET baselines. A multilingual extension to five directions without human ratings retains the dimensions and scoring rule, showing positive system ranking correlations with COMET throughout.
Read the full abstract ▾
All authors Ben Yan, Zongyao Li, Xiaoyu Chen, Daimeng Wei, Weidong Liu, Huan Zhao, Chong Li, Yaode Wang, Yuzhe Shang
Computation and Language
Snugi-AI-v2 @ eRisk 2026 Task 2: Early Depression Detection via a Learned Stopping Policy with Sustained Confidence Gate
Yuwen Chiu
We describe the Snugi-AI-v2 submission to eRisk 2026 Task 2, the second edition of contextualized early depression detection from Reddit discussions. Our central contribution is a learned MLP stopping policy trained to directly optimize ERDE50, replacing the fixed and tiered threshold strategies used in all prior eRisk Task 2 submissions. Combined with a sustained confidence gate that commits only after N=3 consecutive rounds of high policy confidence, the system reduces false positives caused by transient emotional posts without sacrificing recall. The pipeline encodes each discussion thread with a frozen MentalRoBERTa model, maps the accumulated representation to a depression probability via an MLP classifier, and delegates the timing decision to the learned policy. Our best run achieves F1 = 0.73 (Run 1) and F_latency = 0.70 (Runs 0 and 3), with a median alert round of 8 out of 500, completing the full evaluation in 1 hour 26 minutes, the fastest among all complete-submission teams. We report a systematic ablation across five runs spanning two encoder variants, four stopping strategies, and three gate values, along with negative results from GRPO policy training, BDI-II post filtering, MentalLongformer encoding, and DeBERTa ensembling. Code: https://github.com/chiuyuwen91/erisk-2026
Read the full abstract ▾