Computation and Language
Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution
Shuhao Li, Fanghua Ye, Wanyu Lin et al. (+2 more)
Speculative decoding (SD) accelerates autoregressive generation by letting a lightweight draft model propose
tokens that a target model verifies in parallel. However, its standard acceptance rule, designed for exact distribution correction, rejects
tokens to which the draft assigns excess probability, even when the target finds them highly plausible: over half of the rejected
tokens lie inside the target's own
nucleus. Since verification dominates the cost of SD, these rejections waste its most expensive operation. We introduce
Nucleus Speculative Decoding (NSD), a plausibility-aware verification rule that accepts a draft
token if it passes the standard test or belongs to the target
nucleus, while keeping SD's residual correction and bonus sampling unchanged. NSD requires no training or extra forward passes and is compatible with any drafting mechanism. Because it preserves SD's recovery rule, its deviation from the target distribution can be characterized exactly: we prove that the single-step total variation error equals the draft's excess probability inside the target
nucleus, which is precisely NSD's gain in acceptance probability. NSD thus trades fidelity for acceptance at an exact one-to-one rate, and we further derive sequence-level fidelity bounds. Across six target models and five mathematical
reasoning and code generation
benchmarks, NSD achieves the highest throughput among all compared methods, with speedups of up to $5.16\times$ over autoregressive decoding and $3.15\times$ over standard SD, while improving accuracy over SD on five of the six models and nearly halving verification time per output
token. Our code is available at https://github.com/EIT-NLP/
Nucleus-Speculative-Decoding.
Read the full abstract ▾
All authors Shuhao Li, Fanghua Ye, Wanyu Lin, Tianyu Yuan, Xiaoyu Shen
Computation and Language · Cross-listed from cs.LG
$\alpha$Transfer: Coefficient Transfer for Efficient Model Merging
Shih-Cheng Huang, Zhi Rui Tam, Chieh-Yen Lin et al. (+3 more)
Model merging offers a promising solution for combining multiple fine-tuned checkpoints into a single model through parameter arithmetic. However, finding optimal merging coefficients requires an extensive search that becomes prohibitively expensive as models scale in both size and number, due to high
memory requirements and combinatorial growth in the search space. We show that, within the same model family, models exhibit highly congruent performance distributions over merging coefficients across different model sizes. This distributional similarity enables a practical paradigm we call \textit{$\alpha$Transfer}: searching for optimal coefficients on a small proxy model, then directly transfer them to larger target models. We verify $\alpha$Transfer across multiple merging methods, model families, and tasks. Experimental results demonstrate a 6$\times$ speedup and 70%
memory reduction on
vision transformers, and a 20$\times$ speedup and 85%
memory reduction on large
language models, while maintaining comparable performance. Our findings establish $\alpha$Transfer as an efficient and generalizable approach to scaling model merging.
Read the full abstract ▾
All authors Shih-Cheng Huang, Zhi Rui Tam, Chieh-Yen Lin, Yun-Nung Chen, Hung-yi Lee, Shao-Hua Sun
Computation and Language
One Step at a Time: Trading LLM Autonomy for Process Predictability
Hans Schabert and Christoph Peters
Organizations automating operational processes need more than a correct outcome: they need to predict how a process will run, know which one actually ran, and inspect it step by step. When an
agent is the executor that predictability is normally lost: the prescribed procedure goes into the system
prompt, and only a final answer comes back. We deliver the procedure step by step over the Model Context Protocol (MCP) instead: a server releases one step at a time, the
agent executes it, and each step returns a structured step_output. This trades autonomy for predictability, and two properties then follow by construction, independent of the executor. The execution path is prescribed before the run, so the process is predictable in advance rather than reconstructed afterwards; and the completed step records form a machine-readable execution log that downstream tooling can audit and optimize step by step. Evaluating 15,475 trials across 13 SOP-Bench domains and four open-weight executors from frontier (Kimi K2.5) to lightweight (Ministral 3 8B), we find step-level delivery makes the executed process predictable and inspectable for every executor, and additionally raises accuracy when the executor is small. Across all four, process adherence rises significantly (76-95% to 95-99%) and ungrounded answers (correct outputs produced without executing the SOP) near-vanish, falling from 2.1-4.5% to 0.2-0.3% of trials (all 95% CIs exclude zero); under
prompt-based delivery, 31-49% of correct answers on know_your_business bypass the SOP entirely, even for the frontier executor. Accuracy is where the executor's capability enters: the lightweight executor gains +6.5pp grounded accuracy because supplying the process externally removes a reconstruction burden it cannot carry, while capable ones trade a small raw-accuracy decrement for a predictable, auditable process.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
ThinkFuse: Trajectory-Aware Test-Time Fusion for Small Reasoning Models
Myunghoon Kang, Jungseob Lee, Jaehyung Seo et al. (+1 more)
Small
reasoning models (SRMs) have shown strong performance on complex
reasoning tasks by generating extended chain-of-thought trajectories, but they often fail to recover once their
reasoning enters an erroneous path. Existing test-time fusion methods rely on local fusion signals to determine when to trigger fusion, which can be misled by transient uncertainty fluctuations and may reinforce unstable
reasoning trajectories. We propose ThinkFuse, a training-free test-time fusion framework that selectively intervenes in unreliable
reasoning segments. ThinkFuse compares segment-level uncertainty shifts with trajectory-level uncertainty trends to identify unstable
reasoning points and fuse auxiliary
reasoning paths into the primary model's trajectory. Extensive experiments demonstrate that ThinkFuse outperforms baselines on mathematical and knowledge-intensive
reasoning benchmarks, with consistent gains across model-family combinations, and remains robust with a smaller primary model. Our analysis shows that ThinkFuse requires fewer fusion triggers and generates fewer
tokens, highlighting the efficiency of selective triggering. Our code is available at https://github.com/js-lee-AI/ThinkFuse.
Read the full abstract ▾
All authors Myunghoon Kang, Jungseob Lee, Jaehyung Seo, Heuiseok Lim
Computation and Language · Cross-listed from cs.AI
Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell
Hochan Son, Kyungdoe Han, Jaehan Koh et al. (+3 more)
Decomposing long-context inference across cooperating
agents bounds the active KV cache per call rather than total evidence, which matters when KV-cache
memory binds. Many such systems add a persistent tier storing and recalling
reasoning traces, usually validated by an ablation reporting an accuracy gain. We measure both on one three-tier
agent architecture. Decomposition delivers: peak KV working set of 14.3 MiB per query against 35.5 and 35.3 MiB for single-pass and
retrieval-augmented baselines. The persistent tier does not: across eight controlled
dataset pairs at n=100 per arm it costs +0.368 MiB [+0.167, +0.590] of peak cache and produces no detectable accuracy change (+0.015, 95% CI [-0.011, +0.046]). We argue the null is structural: single-question
benchmarks supply each item with its own evidence and score it independently, and correctness requires resetting stored traces between conditions, so recall has nothing informative to retrieve. Reaching it took four measurement corrections – three inflating the apparent benefit, the fourth making an effect that size look resolvable – none visible in the results table. We give the conditions an
agent-
memory ablation must satisfy and detection procedures that need no knowledge of the specific defect.
Read the full abstract ▾
All authors Hochan Son, Kyungdoe Han, Jaehan Koh, Xiaowu Dai, Wenlu Xu, Guang Cheng
Computation and Language · Cross-listed from cs.AI
Quantization Effects on Tool-Failure Recovery Vary Across Prompts and Evaluation Designs
Yuhe Hu
Post-training quantization reduces the cost of deploying
language-model
agents, but its effect on recovery from temporary tool failures can depend on how recovery is evaluated. We compare 8-bit and 4-bit variants of Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct on twenty deterministic tool-use tasks and five
prompts. The 8-bit-4-bit recovery comparison changes direction across
prompts and evaluation targets. On tasks that both variants complete without faults under the same
prompt, the difference ranges from 0 to +20.2 percentage points for Llama and from -50.0 to +35.0 points for Qwen. Full-pipeline point estimates favor 8-bit Llama under all five
prompts, whereas the Qwen comparison changes direction across
prompts. The evaluation target can also reverse the result. For Llama under one
prompt, scoring each variant only on its own clean-passing tasks favors 4-bit by 17.5 points; scoring the same tasks for both variants gives no difference, while scoring the full pipeline favors 8-bit by 28.3 points. Executor leniency is a third such choice. Rescoring the same logs with strict output parsing, which 8-bit Llama violates far more often than 4-bit Llama under that
prompt, turns that +28.3 into -15.0 while leaving Qwen essentially unchanged. These findings show that one
prompt, one screened task set, and one scoring policy do not establish a stable conclusion about quantized-
agent robustness. Evaluations should compare variants on matched tasks, report full-pipeline success for deployment decisions, state the scoring policy, and quantify uncertainty across tasks rather than injected fault sites.
Read the full abstract ▾
Computation and Language
APEX: Speculate smarter, not deeper
Manvi Jha, Zach Zhang, Zhichao Xu et al. (+3 more)
Speculative decoding reduces large
language model inference latency by drafting multiple
tokens before target-model verification, but its effectiveness depends on both the proposal mechanism and draft depth. Fixed configurations cannot respond to changes in predictability, repetition, and acceptance during generation, so deeper drafting can increase wasted computation without proportional speedup. We introduce APEX, a learned controller that balances decoding speed and draft-
token waste through request-level expert selection and block-level depth adaptation. APEX-Router selects among EAGLE-3, n-gram, and draft-model speculation for each request, while APEX-Depth adjusts draft length at each verification block using causal decoding signals and recent verifier feedback. APEX models accepted draft length as censored survival feedback, learning position-wise rejection hazards, block execution costs, and an action utility that balances throughput, accepted progress, and wasted
tokens. This allows the controller to adapt speculation while retaining the target model's verification procedure. We integrate APEX into vLLM and evaluate it with Qwen3-8B across six workloads, achieving up to 5.24X speedup over autoregressive decoding. Across the aggregate evaluation, APEX-S achieves 4.27X speedup, while APEX-B achieves 3.27X speedup with a 41.0% relative reduction in wasted-
token percentage compared with fixed n-gram speculation at k=16, providing distinct operating points for balancing acceleration and draft-
token utilization.
Read the full abstract ▾
All authors Manvi Jha, Zach Zhang, Zhichao Xu, Linbo Liu, Sai Muralidhar Jayanthi, Vinayak Arannil
Computation and Language
Reading, Not Manipulating: Leveraging Router Logits for Multimodal Safety in MoE Vision-Language Models
Ziyuan Yang, Wenxuan Ding, Shangbin Feng et al. (+1 more)
Vision-
language models (VLMs) face compositional safety risks where harmful intent emerges from the interaction between visual and textual inputs. As mixture-of-experts (MoE) VLMs become increasingly common, recent work has explored various safety interventions, including prompting, supervised fine-tuning, and routing-based expert steering. However, these methods show inconsistent improvements across models and evaluation distributions, and the intervention into model behavior or internal states introduce safety-utility tradeoffs by over-refusal. Rather than manipulating internal states to steer model behavior, we instead ask whether routing states can serve as diagnostic signals for
multimodal safety. We find that router logits indeed provide highly predictive signals of whether a
multimodal input is safe or not. Motivated by this observation, we introduce a lightweight router-logit safety detector that reads out routing signals during
prompt prefill and identifies unsafe requests before generation, without modifying model parameters or expert routing. Across Qwen3-VL and Kimi-VL, the proposed detector substantially reduces safety errors on the HoliSafe
benchmark and resoundingly generalizes to out-of-distribution safety
benchmarks featuring different safety patterns, including MISHard and MM-SafetyBench. The success of the proposed router-logit detector also suggests a broader perspective on model internals: rather than focusing only on manipulating internal components to steer behavior, simply reading naturally emerging signals and linking them to an external safety mechanism can provide a simple, effective, and non-intrusive complement to existing safety interventions.
Read the full abstract ▾
All authors Ziyuan Yang, Wenxuan Ding, Shangbin Feng, Yulia Tsvetkov
Computation and Language · Cross-listed from cs.LG
TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models
Xin Wang, Hao Yu, Zhengyang Zhuge et al. (+9 more)
Reinforcement learning (RL) for post-training large
language models (LLMs) incurs substantial computation and
memory overhead during rollout generation, which motivates low-precision rollout for efficient RL training. However, existing FP4 RL methods suffer from a key limitation: they primarily optimize quantization accuracy on the training and rollout paths independently rather than directly reducing the discrepancy between the two quantized execution paths. In this work, we propose TRACE (Train-Rollout Quantization
Alignment via Compact GuidancE), an FP4 quantization framework for RL training of Mixture-of-Experts (MoE)
language models that addresses the limitation of existing FP4 RL methods. TRACE incorporates rollout-guided quantization-aware training that uses rollout-side quantization outcomes to guide training-side FP4 rounding decisions, directly reducing train-rollout discrepancy. Moreover, TRACE adopts an efficient quantization-information caching scheme that selectively retains mantissa and scale information from deeper layers to reduce the storage and communication overhead introduced by rollout guidance. We evaluate TRACE on four large-scale MoE
language models across
reasoning, coding, and long-horizon RL tasks. Our results demonstrate that TRACE enables joint FP4 weight/activation and FP4 KV-cache rollout with RL performance comparable to BF16 rollout, while achieving up to 5.4xrollout speedup and strong final FP4 performance compared with post-hoc FP4 quantization of BF16-trained policies.
Read the full abstract ▾
All authors Xin Wang, Hao Yu, Zhengyang Zhuge, Bochao Mao, Zheng Li, Junda Feng, Yuyan Luo, Yi Zhang, Yizhong Cao, Mi Zhang, Dayiheng Liu, Jianwei Zhang
Computation and Language
No Transformer Beats Six Covariates: Long-Horizon Prediction of Depressive Symptoms from Childhood Essays
Daniel Kua, Emrul Hasan, John-Jose Nunez et al. (+1 more)
Natural
language processing (NLP) models can detect depression-related
language in text written near the time symptoms are measured, but whether pretrained
transformers can predict depressive symptoms from text written twelve years earlier is largely untested. In the National Child Development Study, a British birth cohort, we predict probable depressive symptoms at age 23 from essays the same people wrote at age 11. Our baseline, a logistic regression on six childhood
covariates, outperforms every text model that sees only the essay: seven fine-tuned
transformers, a bag-of-words model, frozen
embeddings and four zero-shot large
language models. Its area under the receiver operating characteristic curve (AUC-ROC) is 0.737 against 0.670 for the best
transformer on the primary seed, and no added text score detectably raises the baseline's AUC-ROC. None of the five domain-pretrained
transformers detectably beats its general-domain control after Bonferroni correction. For long-horizon prediction, the baseline remains the model to beat.
Read the full abstract ▾
All authors Daniel Kua, Emrul Hasan, John-Jose Nunez, Frances Chen
Computation and Language
From Evidence to Action: How Tool-Using Agents Fail
Hongzhan Lin, Shidong Cao, Ziyang Luo et al. (+3 more)
Tool-using
agents make consequential changes to external state, yet correct outcomes do not guarantee that their actions were supported by evidence established beforehand. We study where this evidence-to-action chain breaks as
agents move from deciding whether to act to executing single actions and dependent workflows. Across ten model-harness configurations, strong static action assessment can coexist with much weaker interactive execution. Failures often begin before execution:
agents stop with incomplete investigation or act before required evidence is established. Once required evidence is obtained, single-action execution is usually reliable, while multi-action workflows additionally expose unresolved prerequisites and incomplete execution. For this analysis, we introduce SafeActBench, comprising 656 cases across six operational domains and five protocols that progress from static action judgment and investigated non-action to single- and multi-action workflows. A provenance-bound Evidence Ledger and deterministic trajectory evaluator track what information was established, when actions occurred, and whether downstream dependencies were satisfied. These results show that failures arise not only from missing information, but also from how
agents use established evidence when deciding and executing actions.
Read the full abstract ▾
All authors Hongzhan Lin, Shidong Cao, Ziyang Luo, Wenhao Chai, Mong-Li Lee, Wynne Hsu
Computation and Language · Cross-listed from cs.IR
Learning to Retrieve via Reinforcement Learning in Embedding Space
Qi Liu, Fengming Liang, Yiqun Chen et al. (+2 more)
Dense
retrieval models are typically trained with contrastive objectives that learn effective representations but do not directly optimize
retrieval metrics or downstream task performance. To address this problem, we introduce RELER (REinforcement LEarning for
Retrieval), a reinforcement learning framework that enables existing
embedding models to learn to retrieve directly in
embedding space and align to task-specific rewards. We train RELER by sampling unit-length query and document
embedding actions from von Mises-Fisher (vMF) distributions centered on normalized encoder outputs, scoring the resulting
retrieval or downstream outcomes as rewards, and updating the encoder with REINFORCE using a leave-one-out baseline (RLOO). As exploration in the high-dimensional
embedding space is prone to sampling noise, we further propose conditional-mean projection (CMP), which projects each sampled
embedding onto the low-dimensional subspace spanned by its encoder output and the candidate
embeddings it is compared against, reducing noise in the policy gradient while preserving its expectation. We evaluate RELER on BRIGHT, a
benchmark with
reasoning-intensive queries that remain challenging for existing
embedding models. RELER consistently outperforms InfoNCE and LambdaLoss in average nDCG@10 when post-training BGE-M3 and Qwen3-
Embedding backbones. We further evaluate downstream utility through
retrieval-augmented generation (RAG), where we adapt only the query encoder while keeping the document index and generator fixed. Across seven QA
datasets, jointly optimizing
retrieval and answer rewards improves both average
retrieval performance and answer quality in RAG.
Read the full abstract ▾
All authors Qi Liu, Fengming Liang, Yiqun Chen, Erhan Zhang, Jiaxin Mao
Computation and Language
SanSi: A Looped Typed Decision Model for System 1.5 Thinking
Shuyu Gan, Young-Jun Lee and Dongyeop Kang
Typed decision models answer a declared question without generating text: a decision head returns a probability for each of the declared options in a single forward pass. A single pass is fast, intuitive System 1 thinking. We study what lies between one pass and generated
reasoning: looping, in which the same layers are recursively applied several times before one typed readout. Each loop lets the model revise its hidden state before it commits to an answer, without generating a
token; we call this System 1.5 thinking. We propose SanSi, which turns a pre-trained looped
language model into a typed decision model. The option probabilities are read after every loop, and every loop is trained with a proper scoring rule, so that one model serves every budget from one loop to eight in a single run. On 10,027 test decisions from 59 sources, SanSi reaches 72.0% accuracy: 13.5 points above a non-looped model of the same shape trained with the same recipe, 5.3 points above a newer non-looped model of its size, and 1.8 points below one with three times the parameters. On two depth-controlled tasks, loops extend the solvable depth beyond the depths seen in training, where the larger single-pass model fails. Used as the judge for policy optimization with reinforcement learning, without gold answers, SanSi raises the share of the generator's correct answers from 37% to 52%.
Read the full abstract ▾
Computation and Language
Does Steering Break Your Model? A Multi-Dimensional Evaluation Suite for LLM Steering Methods
Haotian Yang, Huikang Jiang, Yucheng Wu et al. (+4 more)
Activation steering provides a lightweight and flexible way to control large
language model (LLM) behavior. However, effective steering requires more than inducing the intended behavior: it should also limit unintended changes and remain robust across inputs and training data. Existing evaluations cover these dimensions only in fragments. As a result, the trade-offs between efficacy and side effects have not been systematically characterized. We introduce SteerScope, a two-axis, multi-dimensional evaluation suite that jointly characterizes steering outcomes and method properties through 15 metrics. We score target efficacy and side effects on
language quality, task capabilities, and safety and reliability, and further assess generalization and data dependence through steering-specific metrics for sample efficiency and sample sensitivity. Rather than comparing methods at a single operating point, we characterize the trade-offs between efficacy and side effects. Under matched models, tasks, and evaluation protocols, we
benchmark 23 methods spanning 4 families, including prompting, LoRA, and SFT as baseline methods, and release the suite as an extensible codebase. We find that current activation steering methods do not yet surpass the
Prompt Steering baseline in their overall balance between steering efficacy and side effects: across both model scales, no evaluated activation steering method achieves higher efficacy without incurring greater composite side effects. We further uncover a consistent coupling between steering efficacy and side effects. Under OOD
prompts, target efficacy is often preserved, whereas side effects tend to become more pronounced, particularly through declines in instruction relevance and fluency. Methods also exhibit sharply different sample-efficiency profiles.
Read the full abstract ▾
All authors Haotian Yang, Huikang Jiang, Yucheng Wu, Wen-Jie Jiang, Chenpeng Wang, Yibin Lou, Liangming Pan
Computation and Language
JevCascade: Readout Stability and Inference-Time Compute Allocation for Prefill-Only Decision Models
Ran Li and Lei Chen
We present JevCascade, an inference-time compute allocation framework for prefill-only decision models, built on readout stability and zero-label restricted readout prediction. Prefill-only decision models score every candidate in a menu in a single forward pass and never decode, which makes one call one to two orders of magnitude cheaper than a same-scale generative
language model. We show that this read-out structure carries a testable property. When an intervention changes only the menu and leaves the input text fixed, the post-intervention accuracy is already determined by the cached first-pass distribution. The restriction predictor restricts the pass-1 probabilities to the menu, renormalizes and reads off the argmax, with no labels and no second pass. Across six model families, ten core
datasets (sixteen in total) and two task types, menu-only interventions are predicted to within 4.2 points, exactly for one family. The property is ordinal, not probabilistic, since a probability-level variant of the same
estimator errs by 21.0 points, and class-specific, since same-scale generative models err by 1.6 to 15.8 points and degrade as they grow. It makes inference-time compute decidable before deployment. Uniform extra passes buy calibration but almost no accuracy; at matched cost a confidence cascade beats every scheme that re-asks the same model, and curating the menu beats enlarging the model, a 0.8B model on a curated 5-candidate menu reaching 95.4% on CLINC150 against 80.0% for a 4B model on the full 150-label menu.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SE
When Old Facts Return: Re-Reads, Reverts, and the Limits of Temporal Memory
Neeraj Yadav
A
memory system can retire an obsolete value and later restore it merely because the same old statement appears again. A re-read of an old source and a genuine revert can produce the same observed sequence of values while requiring opposite current answers. We study this ambiguity on 130 extractor-selected atomic transitions derived from software fixes. In the ordinary transition condition, identity-based temporal
memory reaches 98.5% model-judged accuracy with zero observed errors under a literal stale-value proxy. Appending a verbatim re-read of the old statement reduces accuracy to 10.8% and raises the stale-value rate to 88.5%. A guard that refuses to reactivate a previously retired value restores accuracy to 97.7% and reduces that rate to 0.8% in this constructed re-read condition. The guard cannot also recognize a legitimate revert without additional change provenance. Two supporting studies examine exposing retired history to the answer model and supplying current source for changed behavior. An exploratory extraction study over 707 software fixes provides scope context, not a universal coverage estimate. The design implication is to distinguish an observation of a value from evidence that the value changed. Selected inputs, aggregate-only answer records, related-family judges and a post-failure guard evaluation limit the conclusions to the retained experiments.
Read the full abstract ▾
Computation and Language
Detecting LLM-Assisted Vietnamese Writing via Keystrokes under Behavioral Manipulation
Thanh Dong, An Ngo, Minh Dau et al. (+1 more)
We study the robustness of keystroke dynamics for detecting large
language model (LLM)-assisted writing. We introduce a Vietnamese keystroke
dataset capturing realistic writing modes, including bona fide composition, transcription, and paraphrasing. We also define a behaviorally grounded threat model in which users deliberately alter typing patterns. To implement the threat model, we create behaviorally manipulated variants of the data designed to evade keystroke-based detection. We evaluate four keystroke modeling approaches: temporal and rhythmic representations, and sequential representations modeled with a one-dimensional convolutional neural network (1D-CNN) and TypeNet, under user-independent and context-independent settings. The results show that sequential models outperform feature-based approaches in most cases and that keystroke signals encode discriminative information about the writing process. However, detection is not uniformly robust: transcription is reliably identified, while paraphrasing and adversarially manipulated samples are frequently misclassified as bona fide when not explicitly modeled. To address this, we incorporate
adversarial training using behaviorally manipulated data, which substantially improves separability and robustness. These results suggest that keystroke-based detection depends critically on exposure to diverse writing behaviors, and that strong performance under limited conditions does not generalize to realistic or
adversarial settings without targeted modeling.
Read the full abstract ▾
All authors Thanh Dong, An Ngo, Minh Dau, Rajesh Kumar
Computation and Language · Cross-listed from cs.LG
Improving Synthetic Data Generation for Argument Mining via Adversarial Reinforcement Learning
Zhijun Zhang, Qianlong Wang, Keyang Ding et al. (+5 more)
Argument Mining (AM) is fundamentally constrained by the scarcity of high-quality structure-annotated
datasets. While LLMs have shown promise in synthetic data generation, producing synthetic AM data that is both structurally accurate and sufficiently diverse remains a challenging problem. To address this problem, we revisit synthetic data generation for AM from a new perspective and propose a novel
adversarial reinforcement learning framework for data synthesis. The proposed framework jointly optimizes the generator and the discriminator in an
adversarial loop, in which the generator produces structured AM instances, and the discriminator provides learning signals by distinguishing real data from synthetic candidates. This enables the generator to progressively improve both the structural accuracy of generated argument data while maintaining diversity through
adversarial feedback. Extensive experiments demonstrate that the proposed framework consistently improves AM performance on three
benchmark datasets in both full-data and low-resource settings, validating its effectiveness and scalability.
Read the full abstract ▾
All authors Zhijun Zhang, Qianlong Wang, Keyang Ding, Genan Dai, Bowen Zhang, Bin Liang, Ruifeng Xu, Yongsheng Liang
Computation and Language
DLoop: Looped Speculative Decoding
Geonmo Gu, Byeongho Heo, HeeJae Jun et al. (+4 more)
Speculative decoding accelerates autoregressive generation in large
language models. In each drafting stage, a lightweight draft model proposes
tokens that the target model subsequently verifies. With increasingly capable draft models, we find that the target model frequently accepts all
tokens produced in a drafting stage. A verification nevertheless follows each drafting stage, resulting in unnecessary target-model forward passes even when drafting could have continued. Adaptive draft length methods decide during decoding how many draft
tokens precede a verification, but they raise the speedup only for autoregressive draft models. For a parallel draft model, drafting further requires target-model hidden states for draft
tokens that have not been verified. We propose DLoop, a looped form of speculative decoding that adaptively performs multiple drafting stages before verification. DLoop continues drafting while the draft model remains confident and verifies all accumulated draft
tokens together. Loop-aware training keeps the draft model reliable in the additional drafting stages by exposing it to its own hidden states for unverified draft
tokens. By spending additional draft-model forward passes, DLoop reduces the number of target-model forward passes required for verification. Across diverse speculative decoding methods including EAGLE-3, DFlash, Domino, DSpark, and multi-
token prediction modules, DLoop improves the wall-clock speedup by 5 to 41 percent while preserving lossless decoding. Code will be available at https://github.com/naver-ai/DLoop.
Read the full abstract ▾
All authors Geonmo Gu, Byeongho Heo, HeeJae Jun, Yoohoon Kang, Sangmin Lee, Sangdoo Yun, Dongyoon Han
Computation and Language · Cross-listed from cs.AI
Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security
Shaswata Mitra, Raj Patel, Subash Neupane et al. (+3 more)
LLM-based multi-
agent systems (MAS) engage tools, share
memory, and delegate tasks, often encountering
adversarial content. Current defenses for MAS are typically evaluated in isolation, focusing on one attack type at a time, which can lead to costly and hard-to-audit outcomes. This study organizes defenses into five principles, implementing them as DEFER1 (DEterministic-First Enforcement with Residual judgment), which includes a cascade of 28 checks that blocks what it can and refers the rest to a panel of four judges. In independent testing across four domains, attack success rates drop from about 30.0% to approximately 3.0%, with 78% of blocked attacks handled by deterministic checks. Only a quarter of proposals reach the judges in the security-operations domain, illustrating that the rules provide security for attacks violating clear policies, while judges manage those that only misrepresent intent. Both systems have weaknesses, such as a risk-score approval gate that inaccurately approves most attack proposals but few legitimate ones, highlighting the challenges in assessing threats accurately.
Read the full abstract ▾
All authors Shaswata Mitra, Raj Patel, Subash Neupane, Sudip Mittal, Md Rayhanur Rahman, Shahram Rahimi
Computation and Language · Cross-listed from cs.SD
Loud and Clear: Dynamic Activation Steering for Improving Speech Intelligibility in Noisy Environments
Seymanur Akti and Alexander Waibel
Speech becomes less intelligible in noisy environments, and humans naturally adapt their voice to compensate. Inspired by this behavior, we investigate whether a text-to-
speech (TTS) model can be guided to produce more intelligible
speech using activation steering, without retraining. We focus on two characteristics of the Lombard effect: increased vocal effort and hyper-articulation. We introduce a
prompt-relative steering mechanism that prevents steering effects from accumulating during generation while allowing their strength to be adjusted dynamically. Across seen and unseen speakers and multiple
languages, our method produces systematic changes in Lombard-related acoustic features, preserves speaker similarity (89-95%), and reduces WER under background noise by 7-22% at 1 dB SNR. These results show that pretrained TTS models can be dynamically controlled to generate more intelligible
speech without retraining.
Read the full abstract ▾
Computation and Language
Monte Carlo Estimation for KV Cache Eviction
Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer et al. (+5 more)
Most KV-cache eviction methods ask, in effect, which
memory appeared important while reading the
prompt? We instead ask which
memory will matter while answering? Since decoding queries are unavailable at eviction time, prior future-aware methods rely on pseudo-responses or synthetic future-query estimates. We cast fixed-budget future-aware eviction as distributional estimation over plausible model-conditional query trajectories and introduce LORE-KV (Lookahead Output-perturbation with Reliability-weighted Ensembles for Key-Value caches), a training-free method that samples short autoregressive continuations from the frozen target model and uses their response-side query states to estimate
prompt-
token utility.
Tokens are scored by projected leave-one-out
attention-output deletion cost and aggregated across sampled futures with optional trajectory weighting. The temporary continuations are discarded before final decoding, requiring no auxiliary model or training. Ablations isolate the mechanism: at B=128, a single response-side continuation recovers about 89% of the gain over the
prompt-window control, while additional futures provide smaller improvements. At B=128, LORE-KV raises the LongBench average on Qwen2.5-14B from 45.49 to 48.24 (+2.75) and the 16K RULER average on Mistral-7B from 45.20 to 51.05 (+5.85); on Llama-3.1-8B, the RULER margin over AnDPro widens to +8.13 and +11.65 points at 64K and 128K contexts. Gains diminish at larger cache budgets and coexist with task-level regressions. LORE-KV incurs 1.46-2.77x AnDPro's per-sample wall-clock time as a one-time compression overhead across six dense and hybrid-
attention backbones. Project webpage: https://ahsanbilal7.github.io/train_free_kv/
Read the full abstract ▾
All authors Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer, Wajih Hassan Raza, Atta Ul Asad, Young D. Kwon, Michal Valko, Dean F. Hougen
Computation and Language · Cross-listed from eess.AS
A Novel Sentence Stress Detection Framework Leveraging Auxiliary Word-Stress Modeling and Loss Optimization
Tien-Hong Lo, Fong-Chun Tsai, Ting-An Hung et al. (+3 more)
Prosodic stress is a crucial aspect of automatic pronunciation assessment (APA), encompassing both sentence stress detection (SSD) and word stress detection (WSD). SSD highlights semantically salient words that shape discourse meaning, while WSD identifies the primary stressed syllable within each word to ensure lexical clarity. However, most prior work treats SSD and WSD as independent tasks, overlooking their shared reliance on prosodic cues such as pitch, duration, and intensity. To address this gap, we propose an effective SSD approach combining SSD with auxiliary WSD via a novel modeling paradigm. In addition, we introduce a word-span stress regularizer (WSR) that concentrates
token-level SSD probabilities within each stressed word span. Experiments on the TinyStress-15K
benchmark show that the proposed method outperforms strong baselines, with the complete configuration achieving the best SSD result.
Read the full abstract ▾
All authors Tien-Hong Lo, Fong-Chun Tsai, Ting-An Hung, Yu-Hsuan Hsieh, Yao-Ting Sung, Berlin Chen
Computation and Language · Cross-listed from cs.LG
Stateless Language Agents: Scaling Long-Horizon Automated Research
Qizheng Zhang, Changxiu Ji, Isaac Sun et al. (+15 more)
Automated research systems increasingly run LLM
agents over long horizons, but more inference does not by itself produce more progress:
agents replay growing histories, duplicate one another's work, or stop experimenting while
token consumption continues. Yet most evaluations use short budgets or
benchmarks that saturate early, leaving these failure modes untested. We trace these failures to two choices: where research state lives and who decides what to try next. We introduce Stateless
Language Agents (SLAs), built on the principle of stateful search with stateless
agents: no
agent carries its conversation across invocations; instead, the harness owns the research state (candidate solutions and measured outcomes) and reconstructs a fresh and role-specific context for every invocation. What each
agent sees becomes an explicit design choice rather than a history that grows with the run. We implement this principle in the SLA framework, where a stateless Advisor reads harness-summarized evidence across search directions and assigns concrete experiments to parallel Workers. We evaluate SLA against three recent frameworks on software engineering, kernel optimization, and
algorithm design at budgets of up to one billion
tokens. SLA achieves the best final result on every task and reaches the strongest kernel baseline's final performance with over 84% fewer
tokens. Ablations from shared checkpoints show that focused contexts and explicit assignments each contribute to SLA's progress, with effects that can compound over full runs, while the Advisor consumes less than 0.6% of
tokens. These results argue for SLAs, which keep durable research state out of
agent conversations, and show that short evaluation horizons can misjudge research systems and their components.
Read the full abstract ▾
All authors Qizheng Zhang, Changxiu Ji, Isaac Sun, Yuetai Li, Shubhangi Upasani, Sherry Ruan, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Yoonho Lee, Yuzhen Mao, Genghan Zhang, Rulin Shao, Qiuyang Mang, Andy Dimnaku, Changran Hu, Radha Poovendran, Kunle Olukotun
Computation and Language
Yifan Zhang, Jichen Feng, Shihan Qin et al. (+2 more)
State tracking requires an update at every input, but the depth a
Transformer applies to each
token is fixed regardless of sequence length. We introduce the Recurrent Looped
Transformer (RLT), which splits its layers between a parallel causal encoder and a recurrent decoder. At each
token, the decoder merges the encoder output with the previous
token's final decoder state, so the computation path grows with sequence length at a fixed per-
token cost. On six algorithmic tasks, we compare five splits of eight layers with an eight-layer
Transformer over three seeds. Trained on at most 40 bits, two RLT splits generalize parity to 256 bits with 100% accuracy in every seed, while the
Transformer stays at chance. On swap-based $S_5$ permutation tracking at eight times the training length, RLT reaches 97% final-state accuracy versus under 1% for the
Transformer, and accuracy increases with decoder depth. On modular arithmetic beyond the training lengths, RLT reaches up to 93% versus 33% for the
Transformer. Ablations show that these gains depend on the feedback: removing it drops parity and swap-based $S_5$ to chance at every split. Updating the feedback once per four-
token chunk lets known
tokens in a chunk run in parallel and keeps 64-bit parity at 99%, while permutation tracking depends on per-
token feedback: chunking lowers length-64 swap-based $S_5$ from 100% to 20%.
Read the full abstract ▾
All authors Yifan Zhang, Jichen Feng, Shihan Qin, Sun-Yuan Kung, Mengdi Wang
Computation and Language
Large Language Model Orchestration under Heterogeneous Preferences via Explicit Persona Inference
Shuqing Shi, Ziyan Wang, Milind Tambe et al. (+1 more)
LLM orchestration investigates how an orchestrator coordinates a group of autonomous
agents to achieve common goals or maximize collective welfare. The
agents are typically heterogeneous, each holding a private preference that it pursues but does not reveal. Inferring such hidden preferences from behavior has been a subject of long-standing research in game theory and multi-
agent systems. The core challenge lies in maintaining a belief over every
agent's preference and updating it from the
agents' observed actions. Existing LLM orchestrators carry that belief as
prompt text with no explicit update rule. This lets early errors persist and propagate rather than be corrected. We therefore propose HARP (Heterogeneous-preference
Agent oRchestration via Preference inference), a novel framework that moves the belief out of the
prompt. Specifically, HARP maintains one numeric posterior per
agent over a finite set of candidate preferences and updates it in closed form by Bayes' rule. The
language model supplies only actions and per-candidate likelihoods, so estimation is decoupled from its
reasoning. We prove that HARP attains the same $\tilde O(\sqrt K)$
Bayesian regret as explicit joint inference when the factorization is exact. Furthermore, HARP\textsuperscript{+} augments planning with a bonus for actions that distinguish the candidates, so inference continues even when the optimal action is uninformative. Empirical results on three substrates, ranging from payoffs the preferences fully determine, through payoffs that depend on more than them, to scales where explicit joint inference is infeasible, demonstrate that HARP\textsuperscript{+} is the strongest non-oracle method across the class our theory identifies.
Read the full abstract ▾
All authors Shuqing Shi, Ziyan Wang, Milind Tambe, Yali Du
Computation and Language · Cross-listed from cs.AI
LOGIC: An LLM Benchmark for Intent-Grounded Change Impact in Aerospace Electrical Systems
Muhammad Faraz Shoaib, Muhammad Qasim, Raisulhaq Mohammed Rizwan et al. (+3 more)
Aerospace electrical-design revisions can contain multiple genuine changes, although an engineering request may authorize only a subset. Propagating every detected difference can therefore produce overly broad impact reports. We present LOGIC, a controlled
benchmark and evaluation framework in which locally deployable
language models ground a request in a deterministic candidate-change inventory before selected changes are propagated through a typed electrical traceability graph. This separation permits candidate-selection errors to be distinguished from downstream propagation errors. LOGIC contains 168 scenarios, including 144 selection and 24 abstention cases. We evaluate three 7–8B models against intent-agnostic, lexical, and structured-evidence methods, with an oracle-root upper bound. On 96 explicitly anchored selection cases, gate-only structured evidence achieves candidate F1 of 1.0000, compared with 0.9677 for
token-lexical matching. On 12 relational-paraphrase cases,
token-lexical F1 is 0.1772 and gate-only F1 is 0.0000, compared with 0.5000–0.6400 for the large
language models. Model grounding degrades as candidate inventories grow from 4 to 64 changes, while affected-element and typed-path accuracy remain comparatively stable when frozen selections are replayed over graphs of approximately 1K to 100K nodes. Strict evidence gating suppresses false positives but can remove correct semantic selections. An exploratory evidence-empty abstention policy raises strict abstention accuracy to 0.6667 for all three models and reduces unsafe-report rates to 0.1667, while decreasing answerable-case coverage by 16.0–27.1 percentage points. Four of six conflicting requests remain unsafe for each model. These findings support combining literal evidence and
language-model
reasoning with engineering review when intent cannot be established reliably.
Read the full abstract ▾
All authors Muhammad Faraz Shoaib, Muhammad Qasim, Raisulhaq Mohammed Rizwan, Rahmatullah Safdar, Muzammil Adnan Shaik, Abdul Aleem Mohammed
Computation and Language
Two Vectors Replace In-Context Demos: Structured Task Adaptation via Embeddings
Xi Ding, Naichen Shi and Jiawei Zhang
In-context learning (ICL) adapts frozen large
multimodal models (LMMs) to new tasks from a few demonstrations (demos), but re-encodes them at every query, where each demo image adds up to hundreds of visual
tokens. Demo-free methods remove this cost with a compact task state. However, they add it at locations searched per task or at every decoder layer, where task parameters grow with depth. Moreover, inserted
tokens or keys cannot change how the original
prompt divides its
attention within a layer. To address these issues, we propose Structured Task Adaptation via
Embeddings (STAVE), which replaces demos with two task-specific vectors added to existing input
embeddings. Specifically, a readout vector updates the answer-producing
tokens and a context vector updates the other structural
token groups. Both are trained with answer labels on
prompts with and without demos. We justify these design choices theoretically using a first-order analysis of the loss and a margin bound. Extensive experiments on six LMMs and five large
language models show that STAVE matches or outperforms state-of-the-art methods on
multimodal tasks with far fewer task parameters and surpasses 15-shot ICL and prior task vectors on 18 text tasks, all at zero-shot inference cost.
Read the full abstract ▾
Computation and Language
HouseholdBench: Evaluating Large Language Models as Predictors of Household Economic Behavior
Jin Huang, Diego Ferreras Garrucho, Yutong Xie et al. (+4 more)
Large
language models (LLMs) have the potential to meet a key goal in economics: a quantitative model of household decision making, across a variety of settings. Yet existing evaluations cover few surveys and outcomes, and do not study how households adjust to changing economic conditions. We introduce a new evaluation, HouseholdBench, which unites 6 U.S. household surveys and 32 prediction tasks spanning numeric, categorical and probabilistic outcomes, related to consumption, income, labor, expectations, and housing. Using past behavior, demographics and macroeconomic conditions, the tasks test whether LLMs predict behavior, including how households adjust to changes in various policies. We evaluate 13 proprietary and open-weight LLMs against a no-change baseline and a gradient-boosted tree model. Most LLMs outperform the no-change baseline, including for policy response tasks – with the best model lowering error for numeric outcomes by 12.2%. Across most tasks, gradient-boosted trees rank first; leading proprietary LLMs approach their performance, but open-weight models lag. LLMs exhibit systematic over- and underprediction across different tasks. We identify methods that enable a 4 billion parameter open-weight model to match proprietary models' performance: fine-tuning and aggregating 16 predictions per observation. Improvements generalize to policy-response tasks, which are excluded from fine-tuning. We release our
datasets, code, and leaderboard on our website: https://jn-huang.github.io/householdbench
Read the full abstract ▾
All authors Jin Huang, Diego Ferreras Garrucho, Yutong Xie, Walter M. Yuan, Qiaozhu Mei, Chen Lian, Jonathon Hazell
Computation and Language
Quality-Aware Self-Correcting Speech Translation on an Edge Device
Zubair Ajmal Farooq and Diptesh Kanojia
We present a fully offline
speech-to-
speech translation pipeline that runs on a Jetson Nano (4 GB) and corrects its own weak translations without retraining. A Whisper-tiny ASR feeds an Opus-MT translator; multilingual BERT cosine similarity acts as a Quality Estimation (QE) gate, triggering a secondary-pass when confidence falls below a pre-defined threshold $\tau$. We compare three correction methods: QE reranking (M1), Minimum Bayes-Risk decoding (M2), and constrained beam search (M3). On 1,012 FLORES-200 sentences (English-Spanish), M2 at $\tau=0.90$ produces statistically significant improvements over greedy decoding on BLEU (+0.67, p<0.001), ChrF (+0.51, p<0.001), and COMET (+0.0020 at N=3, p=0.002); M1 yields no significant gains, and M3 is significantly worse than baseline (p>0.99). Our central finding is that QE functions effectively as a gate but poorly as a ranker: removing the QE model from candidate selection (M1$\to$M2) does not hurt quality and frees 680 MB from the critical path. Using a gain-to-edit ratio adapted from the post-editing-effort literature, we further show that smaller candidate pools (N=3) yield more surgical corrections with better semantic adequacy, while larger pools (N=10) maximise lexical reward. We release the system and demonstrate live translation across six
language pairs.
Read the full abstract ▾