Computation and Language
Evidence and Intervention: A Coupled Active-Inference Extension of Rational Speech Act Models
Yonghyeon Gwon, Elliot Murphy and Chun Kee Chung
Identical utterance choices can arise from different communicative causes, and identical interpretations can leave different traces in what a listener learns. Rational
Speech Act (RSA) models treat interpretation as inference over speaker meaning, but standard one-shot RSA does not intrinsically distinguish these causal update targets. We develop a coupled active-inference model of dialogue in which a listener's likelihood for an utterance is the policy distribution attributed to the speaker, placing the speaker's expected free energy within the listener's variational free energy. Each utterance is therefore both evidence about and an intervention on a partner. The model separates four updates that RSA approaches typically collapse: inference about the partner's current pragmatic state; learning of partner-specific parameters; prospective evaluation of clarification or repair; and a policy prior shaped by habit and a context-sensitive price of time. Under one-step, exact-inference restrictions, the model recovers the RSA speaker and listener, with RSA as the restricted single-turn limit of the coupled process. Outside these restrictions, the updates obey distinct rules and timescales, predicting which adaptations persist, remain partner-specific, or transfer. Worked examples show audience design reversing after clarification, self-confirming misunderstanding in which both interlocutors have low free energy while disagreeing about reference, and rational closing before uncertainty is resolved. Consistent with critiques of equating natural
language with communication, the model treats communication as a downstream use of linguistic structure and recasts production and interpretation as coupled inference: speaking is both an intervention on a partner and an epistemic action that samples evidence for the speaker's model of that partner.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Hierarchical Credit Assignment for RLVR on Fused Gromov-Wasserstein Geometry
Qi Yu, Ruizhong Qiu, Zhichen Zeng et al. (+6 more)
Reinforcement learning with verifiable rewards (RLVR) has been shown to improve the
reasoning capability of large
language models (LLMs) across diverse
reasoning tasks. However, group-based RLVR methods, such as GRPO, assign a uniform advantage to all
tokens within rollouts of the same outcome. While existing works refine credit assignment of GRPO based on local signals such as
token locations or
entropy, they often fail to capture the global semantic novelty of a
reasoning behavior relative to the current policy. In this work, we propose a hierarchical credit assignment approach for group-based RLVR methods, called HarA, which identifies and encourages semantically novel
reasoning behaviors during RLVR. HarA represents each sampled rollout as a distribution over the hidden states and locations of
tokens, and computes the Fused Gromov-Wasserstein (FGW) barycenters of all rollouts with the same outcome, capturing the internal
reasoning patterns in the latent space under the current policy. The semantic novelty of a
reasoning element can then be measured by its contribution to the FGW distance between the current rollout and the barycenter. While solving the FGW formulation is expensive, we introduce an anchor-guided linearization that turns it into a Wasserstein formulation solvable via the Sinkhorn
algorithm efficiently. By reweighing
token-level advantage of group-based RLVR methods based on the novelty signals, HarA highlights novel
reasoning behaviors at flexible granularities to encourage fine-grained LLM exploration. Extensive experiments across three group-based RLVR methods show that our plug-and-play method effectively enhances the exploration of LLMs, outperforming existing methods across diverse
reasoning benchmarks.
Read the full abstract ▾
All authors Qi Yu, Ruizhong Qiu, Zhichen Zeng, Xuying Ning, Yanjun Zhao, Dongqi Fu, Yinglong Xia, Hong Li, Hanghang Tong
Computation and Language · Cross-listed from cs.AI
ShadowMiner v1 - An Experience Report on Implementing and Measuring a Problem-and-Hypothesis Discovery Engine
Jinhyuk Choi
ShadowMiner v1 is a system that automatically discovers research problems and generates hypotheses from AI papers. It is a nine-stage pipeline. It structures documents into a knowledge graph and finds graph gaps in it - structural blind spots in research. These graph gaps are included in the LLM generation
prompt. Each generated hypothesis is then verified by checking whether it is already covered by existing research, scoring its quality, and checking that the facts it relies on are accurately drawn from its sources. This report does not propose a new generation or evaluation technique. It describes our experience of implementing and applying ideas from prior work, and measuring whether each one actually contributed.
Read the full abstract ▾
Computation and Language · Cross-listed from eess.AS
Factorized Delayed Streams Modeling for LLM-based Streaming ASR
Tatsunari Takagi, Kai Washizaki, Atsushi Kojima et al. (+3 more)
Delayed Streams Modeling (DSM) enables LLM-based streaming automatic
speech recognition (ASR) by aligning acoustic and text streams on a common timeline. DSM adds the padding
token <p> and the word-start
token <w> to the LLM vocabulary and predicts them together with normal text
tokens using the same softmax. We first show that <w> can be removed while maintaining competitive recognition performance. Based on this result, we propose Factorized DSM (F-DSM), which separates the waiting probability for <p> from the distribution over the original LLM vocabulary. This factorization removes ASR-specific
tokens from the text prediction space and allows the large-vocabulary softmax to be skipped on waiting steps. Experiments on the Corpus of Spontaneous Japanese and LibriSpeech show that F-DSM achieves better recognition performance than DSM. It also greatly reduces GPU
memory use while maintaining similar training throughput, provides a small inference speed improvement through softmax skipping, and reduces the degradation in text-only perplexity observed with DSM.
Read the full abstract ▾
All authors Tatsunari Takagi, Kai Washizaki, Atsushi Kojima, Lianbo Liu, Koki Nikaido, Yui Sudo
Computation and Language · Cross-listed from cs.AI
Suppressing Pressure, Amplifying Evidence: Self-Guided Attention Steering to Mitigate Sycophancy and Stubbornness
Yinghao He, Mengyu Xu, Haixiang Sun et al. (+8 more)
Reliable
language models should resist unsupported user pressure while effectively using objective contextual information. However, models may exhibit sycophancy by yielding to unsupported user pressure or contextual stubbornness by failing to update their answers when relevant contextual information warrants revision. Evaluating interventions for these failures separately can obscure whether mitigating one failure exacerbates the other. To assess this trade-off, we introduce CoPE-Bench with six conditions per question: a neutral baseline, correct or incorrect user pressure, contextual information consistent with or conflicting with the neutral answer, and a joint condition combining incorrect claims with conflicting contextual information. To regulate the influence of user pressure and contextual information, we propose SPAE (Suppressing Pressure, Amplifying Evidence), a training-free framework that uses the model's own judgments to identify relevant
tokens, suppressing user pressure and amplifying contextual information through
token-level
attention steering. On average across five backbones, SPAE reduces pressure following by 18.8 percentage points and increases joint-condition updating by 5.5 percentage points relative to the strongest baseline in the main comparison. In two-turn dialogue, it improves joint-condition updating by an average of 13.2 percentage points over the strongest prompting baseline. The source data and codes can be found at https://github.com/03Grant/sycophancy-and-stubbornness.
Read the full abstract ▾
All authors Yinghao He, Mengyu Xu, Haixiang Sun, Donghan Li, Yibo Wang, Lixu Wang, Kezhen Chen, Chi Li, Chunwei Liu, Bharat Bhargava, Chongyang Gao
Computation and Language · Cross-listed from cs.AI
Bidirectional Preference Synthesis: Learning Prompt-Conditioned Preferences from Boundary Failures
Junbo Wang, Lidong Lu, Zhuoqun Li et al. (+4 more)
Correction-based offline preference pipelines commonly treat model failures only as rejected responses under the original
prompt. This supervision is incomplete for boundary failures: responses that violate the given instruction yet coherently satisfy a nearby intent or constraint setting. We introduce Bidirectional Preference Synthesis (BPS), a data-construction method for standard Direct Preference Optimization (DPO) that makes this missing
prompt dependence explicit. For each validated boundary failure, BPS keeps the conventional forward pair under the original
prompt and adds a reverse pair under a synthesized achieved
prompt, so the same response is rejected where it is wrong and chosen where it is right, without changing the DPO objective, training a reward model, or requiring online sampling. On Qwen3-4B-Instruct-2507, BPS preserves original-side pairwise ranking while raising achieved-side ranking accuracy from 6.8% to 62.3% on held-out crossed anchors, with a similar shift under a Kimi-K2.6 cross-teacher probe. A blind human audit supports the intended reverse preference direction, and downstream evaluations show the clearest separation from Forward-DPO in multilingual multi-turn instruction following, with consistent capability-retention patterns on agentic, tool-use, and code checks.
Read the full abstract ▾
All authors Junbo Wang, Lidong Lu, Zhuoqun Li, Guiping Jiang, Xiangyu Wu, Tinghai Zhang, Tong Lu
Computation and Language · Cross-listed from cs.AI
From Latent Space to Jacobian Space: Measuring, Evading, and Training Against Safety-Content Accessibility
Mohammad Mosafer
Output-only safety monitoring sees only the end of a model's computation, yet the model computes its answer before emitting it: what it is internally poised to say is safety-critical. Jacobian-space (J-space) readouts, linear maps from hidden states to the output vocabulary via the model's input-output Jacobian, have been proposed as a window on behaviorally accessible internal content, and a first safety protocol (JADR) showed that danger recognition is readable there. What remains unknown is how this accessible content relates to the latent-space safety geometry studied by representation engineering, and how safety training shapes it. We introduce two quantitative bridges: (i) a transport-amplification profile $A_\ell$, measuring how strongly a latent safety direction is carried toward output space by each layer's Jacobian, and (ii) a paired base-vs-tuned protocol that attributes accessibility to training provenance. Across four small model pairs (135M to 1.5B, three lens-fit seeds), tuned checkpoints concentrate recognition in upper-middle layers, and at 0.5B DPO installs refusal while J-space recognition drops to chance: training can widen the accessibility gap exactly where behavioral safety looks best. A deployment audit closes the loop: monitor-aware GCG-style suffixes suppress the
prompt-side monitor at zero behavioral cost at every scale; only a learned monitor rung resists its own adaptive re-attack; the training-time defense fails its re-attack at every penalty weight; steering shows the amplification profile is descriptive, not causal; and continuous-prefix optimization fails where discrete search succeeds. Safety monitoring, training, and evaluation must operate on accessibility itself, not on outputs alone.
Read the full abstract ▾
Computation and Language
Evaluating Modeling Approaches for Experience-Level Classification in Job Description
Celia Liang, Eddie Wu, Shiqi Wang et al. (+1 more)
This paper investigates the task of predicting job experience levels in recruitment texts, aiming to automatically identify the qualifications required for positions. Unlike traditional text classification, recruitment texts typically possess explicit internal structures, with different paragraphs playing disproportionate roles in conveying experience clues. To address this, we propose a structure-aware Section-Aware BERT approach that segments and encodes key paragraphs (titles, responsibilities, requirements) for integrated modeling, building upon rule-based systems and classical baselines TF-IDF. Simultaneously, we evaluate large
language models under both few-shot and fine-tuning settings on the same
dataset to compare the capability boundaries of different modeling paradigms. Experimental results demonstrate that explicitly leveraging text structure significantly improves experience level prediction performance, particularly in scenarios with ambiguous job titles. Further error analysis reveals systemic challenges in this task, including confusion between Entry and Senior levels and the blurred boundaries of Mid-level positions. This research provides an effective modeling approach and analytical framework for understanding structured recruitment texts.
Read the full abstract ▾
All authors Celia Liang, Eddie Wu, Shiqi Wang, Yonah You
Computation and Language · Cross-listed from cs.LG
Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
Jinyuan Li, Chengsong Huang, Langlin Huang et al. (+4 more)
Self-evolving
reasoning models learn from their own generated questions, yet repeated self-training can lead to performance collapse. In this paper, we investigate why performance deteriorates over successive rounds and how to sustain self-
evolution. Our analysis identifies two recurring quality problems in self-generated questions: invalid questions and repeated variants of the same mathematical questions. First, invalid questions become more prevalent across rounds, and answer-consistency filtering further increases their proportion in training data. Second, existing question diversity controls based on lexical similarity can miss mathematically equivalent questions expressed in different ways, which leads to question diversity collapse in later training rounds. Building on these findings, we introduce R-Quest, which uses question validity and novelty feedback to guide self-
evolution. We first train the solver to recognize and reject invalid questions, then use its judgments to guide questioner rewards and filter solver training data. To avoid question repetition, we use a frozen base model to compare sampled question pairs and provide novelty feedback. Empirically, our method consistently achieves the highest average performance on 12
benchmarks in mathematical
reasoning, general-domain
reasoning, and code generation across two model families. Additionally, R-Quest maintains stable performance gains over ten rounds of self-
evolution, peaking in the final round and outperforming R-Zero by 17.32 points.
Read the full abstract ▾
All authors Jinyuan Li, Chengsong Huang, Langlin Huang, Donghong Cai, Shiping Gao, Yuyi Yang, Jiaxin Huang
Computation and Language · Cross-listed from cs.AI
Language-Conditioned Token and Reasoning Efficiency in Large Language Models: A Paired Cross-Lingual Study Protocol
Genliang Zhu and Chu Wang
Large
language models incur
language-dependent representation and inference costs, but existing comparisons often conflate input
language, assigned observable-trace
language, and answer realization. We specify a prospective paired study that separates these interfaces while holding the semantic item, checkpoint, and answer oracle fixed. The initial design instantiates 240 exactly scored items rendered from templates in English and seven non-English
languages, three distinct-lineage open-weight checkpoints, three trace-
token budgets, 22 input- and trace-
language conditions, a fixed answer reserve, and a separately counted delimiter: 47,520 initial core runs before prospective sample-size selection. RQ1-RQ3 estimate input and trace effects by intention-to-treat with failure-inclusive terminal accounting and test answer realization by cloning a sealed prefix and runtime-native KV state into eight crossed branches. Pre-freeze independent
language review, fixed-form ASCII selectors, and code/surface/solver agreement constrain the realization test. H1-H5 share one Holm family and a global simultaneous component band. A secondary randomized experiment compares one-long-attempt and complete K-short-attempt policies at equal trace allowance under frozen seeds and oracle-blind aggregation; it is a full-policy contrast because answer capacity differs. Outcomes include exact
token spans, correctness, latency, runtime-exposed
memory, and qualified same-host operating-system-reported energy over prespecified hardware rails. The protocol separates tokenizer expansion, observable-trace cost, and answer-realization cost without treating visible traces as internal cognition or operating-system estimates as physical cross-device energy. No confirmatory model outcome is reported; result fields remain disabled until the frozen evidence ledger passes independent verification.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
First-Order Steering: Translating Weight Adaptation into Activation Steering
Sri Pranav Kunda, Alexander Kurz, Tomas Dominik et al. (+1 more)
Activation steering exploits interpretable directions in the residual stream to enable inference-time manipulation of model behavior. Composing steering vectors to apply multiple target behaviors simultaneously is important in various fields-including AI
alignment and safety-but remains a challenge for existing activation steering methods. In contrast, prior work in model merging shows that target behaviors represented by learned weight adaptations can be combined with high accuracy. A method that translates weight adaptations into activation steering vectors could therefore extend prior work in model merging to generate composable steering vectors that better enable simultaneous inference-time behavioral control. For this, we introduce First-Order Steering, a formulation of activation steering as a first-order approximation of weight update
matrices parameterized by a vector of steering strengths, and establish theoretical bounds on the approximation error of first-order steering. We then develop a novel model merging procedure, HeRD-Merging, which minimizes the first-order approximation error terms to enable higher first-order steering accuracy. Together, our method produces steering vectors that control both individual and composed behaviors more accurately than existing activation steering methods. Furthermore, HeRD-Merging matches the performance of conventional model-merging baselines, while producing weight adaptations that admit more accurate first-order steering vectors.
Read the full abstract ▾
All authors Sri Pranav Kunda, Alexander Kurz, Tomas Dominik, Uri Maoz
Computation and Language · Cross-listed from cs.LG
Rethinking Self-Distillation for Multi-Teacher Capability Merging
Roy Xie, Dan Friedman, Feng Nan et al. (+7 more)
Combining capabilities of multiple expert models trained starting from the same base checkpoint has become increasingly common in frontier
language-model post-training. Recent trends suggest that multi-teacher on-policy
distillation (MOPD) outperforms conventional off-policy methods. However, despite the higher inference and environment interaction costs incurred by MOPD, we find that much of its reported accuracy gain is due to certain training design choices and hyperparameter optimization, rather than the
algorithm itself. We conduct a controlled self-
distillation study across two multi-teacher settings, four models, and eleven
benchmarks, comparing off-policy methods, namely supervised fine-tuning (SFT) and soft-label
distillation, to hybrid teacher-prefix
distillation and MOPD. We found that all four methods achieve nearly identical accuracy. However, MOPD uses $14.8$–$23.1\times$ SFT's training GPU-hours. We also revisit four recently published comparisons between on-policy and off-policy
distillation and find that the reported on-policy gains shrink substantially when SFT baselines are trained on rejection-sampled teacher trajectories and use independently tuned hyperparameters. As a training-free alternative, we also find that simple weight-merging methods can recover expert capabilities with minutes of CPU merging time, although their accuracy degrades as model size decreases and task interference increases. Overall, our results question recent gains reported due to MOPD and suggest careful tuning of more efficient off-policy baselines as a viable alternative.
Read the full abstract ▾
All authors Roy Xie, Dan Friedman, Feng Nan, Yukun Huang, Zhichao Xu, Chengjiu Zhang, Jun Xu, Manaal Faruqui, Vivek Rathod, Bhuwan Dhingra
Computation and Language
AI-Enabled Quality Assurance for Multiple-Choice Assessment Items
Steven Moore and Nicholas Diana
Generating multiple-choice questions is increasingly scalable, but establishing their assessment quality remains difficult. We present a focused narrative review of automated item-writing flaw detection, revision, psychometric screening, and NLP
benchmark auditing. Database searches, citation
retrieval, and nominated sources yield fourteen research reports reviewed in full text. We distinguish surface checks from content-sensitive judgments and map a 19-criterion rubric to detection methods and reported evidence. High label-level accuracy often coexists with weak positive case detection, while rubric definitions and reference standards vary. Revision evidence is mixed, and the associations reported in prior work do not establish the effects of repair. We propose evaluating quality assurance as a sequence of independently validated decisions, with criterion-specific reporting, calibrated human review, and outcome-based assessment of revisions.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
Multimodal Dual-Encoder Retrieval for Automated ICD Coding
Abhinav Bohra and Anuj Bohra
Accurate International Classification of Diseases (ICD) coding is crucial for large-scale clinical research, documentation, and billing. There are three primary problems with current ICD prediction methods: (1) They are unable to comprehend
multimodal patient data because they rely on either structured EHR data or unstructured clinical notes. (2) They also struggle with scalability to a larger amount of ICD codes (9K+ codes in ICD-9), as traditional classifiers need dense output layers and often do not generalize well to long tail rare diseases. (3) They lack transparency for clinical use. To address these challenges, this research proposes a two-stage framework that first retrieves ICD codes using a
multimodal dual-encoder
retrieval model, where structured and unstructured patient data are integrated through gated fusion. The second stage refines the top-k retrieved candidates with an LLM-based re-ranker that provides ranked codes with clinically relevant explanations. Our experiments show that the proposed approach improves Micro-F1 and Precision over a
multimodal dual-fusion classifier baseline. These improvements demonstrate that combining a gated
multimodal retrieval system with LLM-based re-ranking is a practical alternative to dense multi-label classification for automated ICD coding.
Read the full abstract ▾
Computation and Language
Playing social deduction games with reinforcement fine-tuned large language models
Lingzhe Zhang, Yunpeng Zhai, Tong Jia et al. (+7 more)
Reinforcement fine-tuning (RFT) is increasingly used in applications where large
language models (LLMs) interact with humans and other
agents. Here we use social deduction games to study how RFT changes LLMs' social behaviour. We let fine-tuned and base LLM
agents play hidden-role games that require hidden-state inference, social reading and vote steering. Our results show that LLM
agents do not reliably acquire social-deduction ability by directly optimizing terminal win–loss outcomes, suggesting that final game results provide a sparse and noisy signal for socially interactive learning. However, RFT is particularly effective at improving social reading, including tasks that require
agents to infer hidden roles from public discussion, update beliefs over time and predict other
agents' future decisions. We further show that RFT can also improve social influence, including tasks that require
agents to steer votes, team approvals and collective decisions, although these gains depend more strongly on behaviourally specific rewards and structured interaction settings. Finally, we show that LLMs' ability to play social deduction games can be further improved through multi-
agent social-cognitive reinforcement fine-tuning, which combines social-reading and social-influence signals during same-side multi-
agent training. These learned behaviours also receive more favourable human evaluations of strategic competence, persuasiveness and social usefulness. Together, these results enrich our understanding of how RFT changes LLMs' social behaviour and provide a step toward a behavioural learning theory for machine social intelligence.
Read the full abstract ▾
All authors Lingzhe Zhang, Yunpeng Zhai, Tong Jia, Kening Zheng, Chiming Duan, Minghua He, Zhaoyang Liu, Bolin Ding, Philip S. Yu, Ying Li
Computation and Language
Conformal Prediction with Paraphrase-Aware Scoring for LLM Uncertainty Quantification
Jiayi Xin, Evan Qiang, Zihan Zhu et al. (+3 more)
Uncertainty quantification (UQ) for large
language models (LLMs) aims to provide reliable measures of predictive confidence, yet current methods are often unstable under meaning-preserving perturbations. Semantically equivalent paraphrases can induce substantial variability in predictive confidence, even for methods with formal guarantees, such as conformal prediction. To address this issue, we propose a paraphrase-aware UQ framework robust to semantic rewordings. Our approach trains a lightweight proxy model on LLM hidden states and aggregates its predictions across paraphrases to construct label-wise nonconformity scores. Under score exchangeability, conformal calibration retains marginal coverage. This guarantee can also hold under test-only rewording, provided that the paraphrase pipeline satisfies an additional distributional
alignment condition. We evaluate three settings (normal, fully reworded, and semi-reworded) which apply rewording to neither
dataset, both calibration and test
datasets, or only the test
dataset, respectively. Across seven multiple-choice QA
benchmarks and multiple model families, our method produces compact prediction sets with empirical coverage generally near the nominal target, even in the semi-reworded setting. Ablation studies show that the learned proxy accounts for most of the reduction in set size, while paraphrase-augmented training and inference-time aggregation improve stability under rewording. Code is available at https://github.com/Raina-Xin/PA_Score.
Read the full abstract ▾
All authors Jiayi Xin, Evan Qiang, Zihan Zhu, Xiang Li, Weijie J. Su, Qi Long
Computation and Language
Can LLMs Separate Pasted Artifacts from User Speech? Absorption at Unmarked Prompt Seams
Sugam Panthi, Muhaiminul Yeamin and Rabab Abdelfattah
Large
language models (LLMs) receive each user message as plain text, even when it combines text from different sources. For example, a user may paste text into a
prompt and keep typing a comment directly below it. We study absorption: a phenomenon where the model treats a trailing user comment as part of the pasted text, returning it inside the edited text. This happens even though the user did not intend the comment to become part of that text. Existing instruction-data separation
benchmarks tell the model which text is instruction and which is data, then test whether it obeys that separation. They do not test harmless user
speech following an unmarked paste. We introduce SEAM, a controlled
benchmark of 300 editing examples. Each example is tested under six matched conditions that vary how the boundary between pasted text and later user
speech is expressed. Across 20 models, absorption at a bare newline ranges from 7.7% to 66.7%. Adding a blank line does not significantly reduce absorption in any model, while boundary markers reduce it in 19 of 20 models. Comments that fit the pasted text, such as a code comment typed after code, are absorbed significantly more often in 17 of 20 models. Models often fail to separate pasted material from later user
speech, and explicit boundaries reduce but do not remove this failure.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Clean: Second-order LLM Training at Linear Memory Cost via Nyström Sketching
Beheshteh T. Rakhshan, Sahar Rajabi, Maziar Sargordi et al. (+3 more)
Training large
language models (LLMs) entails a fundamental trade-off:
memory-efficient optimizers such as Adam discard cross-parameter
curvature, whereas full-
curvature methods such as SOAP can accelerate convergence at prohibitive
memory costs. We introduce Clean, a
memory-efficient and full-
curvature optimizer designed to resolve this bottleneck. Clean leverages the randomized Nystrom method to accurately approximate the left and right preconditioners in SOAP, and to reduce the optimizer's
memory complexity from quadratic to linear in terms of model dimensions. We subsequently reintegrate the off-subspace components to capture
curvature information beyond the low-rank approximation, preserving rich
curvature at minimal
memory cost. We further propose Q-Clean, a low-precision variant that aggressively compresses optimizer states. Q-Clean reduces optimizer
memory consumption by over 50% compared to
Muon when pre-training a LLaMA-1.3B architecture, all while maintaining strong and competitive predictive performance. Notably, Clean operates with a smaller optimizer-state footprint than standard AdamW while reaching AdamW's final performance 26% faster in wall-clock time. Furthermore, our methods uniquely enable the pre-training of a 13B-parameter model on a single 80GB GPU, providing a scalable, efficient, and accessible approach to large-scale model optimization.
Read the full abstract ▾
All authors Beheshteh T. Rakhshan, Sahar Rajabi, Maziar Sargordi, Shikai Fang, Guillaume Rabusseau, Sirisha Rambhatla
Computation and Language · Cross-listed from cs.AI
Language Model Activations Inhabit Privileged Error-Correcting Basins
Matthew Finlayson, Francisco Pernice, Eric Todd et al. (+16 more)
Language models exhibit remarkable robustness, continuing to produce coherent text even when their activations are perturbed by interventions like linear steering. We hypothesize that this robustness is a result of passive dynamics, i.e., constraining mechanisms in the forward pass that funnel activations toward "good" regions that produce coherent outputs. To investigate these hypothesized error-correcting mechanisms, we probe the geometry of
language model activation space by observing the action of model layers on low-dimensional curves. In doing so, we discover that model activations occur within a cluster of distinct attracting basins, which differentiate natural activations geometrically from distributionally similar synthetic activations. Applying this lens to
language model steering, we observe feature-specific basins along semantic steering directions, and find that steering moves activations between these basins. To demonstrate the active role of this geometry in neural computation, we show that adaptively modulating steering strength to transport activations across basins improves inter-
language steering, significantly increasing the probability of sampling
tokens from the target
language compared to fixed-strength steering. Our findings establish analysis of activation space geometry as a promising approach to interpreting and controlling
language models.
Read the full abstract ▾
All authors Matthew Finlayson, Francisco Pernice, Eric Todd, Amir Zur, Daniel Wurgaft, Fenil R. Doshi, Vasudev Shyam, Matt Feiszli, Satchel Grant, Lucius Bushnaq, Tal Haklay, Usha Bhalla, Matthew Kowal, Thomas Fel, Jack Merullo, Atticus Geiger, Xiang Ren, Owen Lewis, Ekdeep Singh Lubana
Computation and Language · Cross-listed from cs.HC
DimSteer: Steering LLM Authoring with Automatically Discovered Stylistic Controls
Ajit Mallavarapu and Ziwei Gu
Large
language model writing interfaces often make users steer outputs by repeatedly articulating desired changes in natural
language. Yet writers may recognize useful stylistic directions only after seeing alternatives, making revision recall-heavy. We present DimSteer, an authoring interface that samples
prompt-local completions, discovers high-variance activation-space axes of variation, labels them, and exposes them as sliders with pole previews, diff comparison, and reset controls. Users can manipulate discovered dimensions, reducing the need to reformulate
prompts for each stylistic adjustment. In a within-subjects study with 16 participants against a matched
prompt-only baseline, DimSteer reduced mental demand, effort, and frustration while preserving comparable perceived success. Participants valued the surfaced dimensions, yet 15 of 16 disagreed that they would have thought to request the same changes in a
prompt. Results suggest
prompt-local controls can shift LLM authoring from recall-based prompting toward recognition-based exploration and direct manipulation, while preserving prompting for open-ended edits.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Principled Top-$k$ Selection for Language Models with Hybrid Gradients
Xuchen Gong, Junfei Sun and Tian Li
Selecting the best $k$ items out of $m$ candidates is a critical component of modern large
language model systems, such as document selection in
Retrieval-Augmented Generation (RAG) and expert routing in Mixture-of-Experts (MoEs). However, training these selection modules remains challenging due to weak gradient signals and suboptimal exploration-exploitation tradeoffs. Furthermore, prior works often rely on heuristics, lacking principled objectives and approaches that explicitly model and solve the top-$k$ selection problem. In this work, we propose a principled objective for training selection modules, whose gradient naturally provides richer training signals in a hybrid form—containing a supervised-gradient component and a policy-gradient component. We show that the selection problem becomes harder as $m$ increases, and our
algorithm converges at rate $O(1/\sqrt{T})$, with the optimal upper bound achieved by balancing between bias and variance. Practically, we apply our method to a set of tasks involving top-$k$ selection, including synthetic regression problems, RAG, and MoE systems, showing that our method outperforms the baselines in next-
token prediction perplexity and QA accuracy.
Read the full abstract ▾