Computation and Language · Cross-listed from cs.FL
Regularity as seen by Alice and Bob
Omid Yaghoubi, Mikołaj Bojańczyk, Aliaume Lopez et al. (+1 more)
The goal of this paper is to propose a unifying model for Nerode-style characterizations of regularity across functions with different output domains. Building on Hauser's work in communication complexity, we generalize the setting by relaxing the computability assumptions and allowing non-Boolean output domains. We consider functions of type $\Sigma^* \to \domain$, where $\Sigma$ is a finite alphabet and $\domain$ is an arbitrary domain. For several domains, we show that the model coincides with known models of computation. We further
conjecture that an analogous correspondence holds for other domains that currently lack a Nerode-style characterization of regularity, and we provide ample supporting evidence. In the model, an input string $w$ is split as $w = w_1 w_2$ and distributed between two cooperating parties, Alice and Bob, who exchange a constant number of messages to compute the value of the function. Each message is either an element of the output domain or a signal drawn from a finite set of signals, and the parties must produce the correct output for every admissible split $w = w_1 w_2$. We further extend the framework to infinite alphabets in the setting of nominal sets, and investigate its expressiveness on
languages of words with atoms.
Read the full abstract ▾
All authors Omid Yaghoubi, Mikołaj Bojańczyk, Aliaume Lopez, Rafał Stefański
Computation and Language
Post-Training Shifts Confidence: A Three-Stage Analysis of How SFT, RL, and OPD Shape CoT Calibration
Shuhao Li, Guodong Du, Anhao Zhao et al. (+3 more)
Large
language models have made strong
reasoning gains through supervised fine-tuning, reinforcement learning, and on-policy
distillation, yet these post-training methods are usually evaluated only by final-answer accuracy. We study how they reshape confidence during
reasoning. We introduce a three-stage calibration framework that evaluates confidence before, during, and after chain-of-thought generation, corresponding to difficulty estimation, early termination, and answer aggregation. Through a controlled comparison on mathematical
reasoning benchmarks, we find that OPD provides the most useful pre-
reasoning confidence, SFT gives the strongest online signal for early stopping, and RL produces the most reliable trace-level signal for aggregation. We further show that confidence reliability is position-dependent: RL confidence becomes informative after a path-commitment phase, while OPD confidence is useful early but can become inversely calibrated later. Based on this observation, we propose PosConf, a position-aware confidence strategy that uses confidence only from reliable relative-position intervals. PosConf improves RL answer aggregation by 6.1 points over majority voting and consistently improves OPD early stopping under tight
token budgets, with gains up to 4.3 points by avoiding its later inverse-calibration region, showing that confidence in
reasoning models should be used both stage-wise and position-awarely. Our code is available at https://github.com/EIT-NLP/Post-Training-Calibration.
Read the full abstract ▾
All authors Shuhao Li, Guodong Du, Anhao Zhao, Wanyu Lin, Tianyu Yuan, Xiaoyu Shen
Computation and Language
Self-supervised Speech Comparison for L2 Phone, Rhythm, and Intonation Scoring
Stephen McIntosh, Reuben Smit, Daisuke Saito et al. (+2 more)
L2
speech assessment has traditionally focused on phonetic assessment, leaving the scoring of suprasegmental features such as rhythm and intonation underexplored. Moreover, assessment methods often require training with labeled L2
speech data, making them difficult to apply in low-resource settings. We investigate whether DTW over self-supervised WavLM representations can provide a text-free framework for assessing phonetic accuracy, rhythm, and intonation in English and Japanese L2
speech. Results show that a basic DTW-based approach that compares learner
speech to native templates exceeds human agreement on holistic and sentence-level phonetic scoring. For rhythm, we introduce methods that measure the degree of warping in the DTW
alignment path; our best method approaches human-level performance. For intonation, we combine DTW distance over prosodic residuals with pitch and intensity features, but performance remains more modest on some tasks. Our results point to self-supervised representations as a promising, text-free basis for multi-aspect pronunciation assessment.
Read the full abstract ▾
All authors Stephen McIntosh, Reuben Smit, Daisuke Saito, Nobuaki Minematsu, Herman Kamper
Computation and Language · Cross-listed from cs.CV
Groc-PO: Grounded Context Preference Optimization for Truthful Multimodal LLMs
Zhixiao Zheng, Zheren Fu, Zhiyuan Yao et al. (+3 more)
Despite the rapid progress of
Multimodal Large
Language Models (MLLMs), they still suffer from untruthfulness issues, such as visual
hallucinations, content fabrication, and unfaithful
reasoning, which substantially undermine their faithfulness and practical utility.
Alignment methods based on human preference, such as Direct Preference Optimization (DPO), have been widely adopted to address these issues. However,
multimodal reasoning errors often propagate across stages, and final-answer errors can often be traced to mistakes in early grounding stages, yet standard DPO typically applies preference optimization at the final-answer level. This credit-assignment challenge means that supervision for early grounding stages is indirect rather than stage-specific, making it difficult to suppress error propagation arising from grounding drift and context inconsistency. To address this, we propose Grounded Context Preference Optimization (Groc-PO), a grounded preference optimization framework for MLLMs. We further construct the Grounded Context Preference
Dataset (GCPD), organizing multi-stage preference samples around three stages of Object Grounding, Contextual Grounding, and Grounded
Reasoning, to capture the formation, integration, and utilization of grounded context. By introducing more explicit preference supervision over multiple grounded stages, Groc-PO strengthens context-dependent
reasoning and mitigates cross-stage error propagation. Extensive experiments show that, compared with standard DPO and other strong baselines, Groc-PO achieves improved performance in
hallucination mitigation, faithful
reasoning, and overall reliability, supporting the value of more explicit grounded supervision for trustworthy
multimodal reasoning.
Read the full abstract ▾
All authors Zhixiao Zheng, Zheren Fu, Zhiyuan Yao, Chunxiao Liu, Dongming Zhang, Zhendong Mao
Computation and Language
The Test Oracle Problem in Synthetic LLM-as-Judge Corpora: Disappearance, Distortion and a Validation Protocol
Serkan Ball{\i}
Studies of bias in LLM-as-judge systems typically build synthetic corpora by prompting an LLM to generate a hallucinated answer to pair with a factual one, then presenting both to a judge. We report a case in which this generation step silently failed, and use it to argue that the failure mode is structural rather than incidental. In a
multilingual (Turkish/English) faithfulness-judgment corpus, a decoding-budget parameter shared between judging and generation calls truncated one producer's hallucinated answers to a few words. The resulting items produced a large, statistically robust effect: a 32-point cross-lingual collapse in one judge's selection accuracy, replicated from N=50 to N=500, explained by a three-layer mechanistic account, and confirmed by a controlled producer-swap experiment, none of which was real. The effect vanished to ceiling once the shared parameter was corrected, and only manual reading of the raw generations, not any aggregate statistical check, exposed the fault. A second measured bias (Markdown-formatting preference) was not fabricated but distorted by the same fault, its magnitude and in one case its sign shifting with stimulus length, a mode aggregate metrics cannot distinguish from the first. We frame the underlying vulnerability using the test oracle problem: corpora whose negative examples are LLM-generated carry no mechanical way to verify item integrity, while corpora built by deterministic perturbation of a gold answer carry an item-level oracle for free. A positive control supports this claim directly: an analogous fault injected into a minimal perturbation-based corpus is caught with 100% accuracy by a zero-cost, zero-human gold-to-negative string comparison. We close with a validation protocol, derived from our own case, for analysts working in the oracle-less regime that we argue describes most contemporary
multilingual LLM-as-judge corpora.
Read the full abstract ▾
Computation and Language
HarnessBank: Semantic Gene-Bank Search with Gated Verification for Agent-Harness Self-Evolution
Xiaotian Luo, Dizhan Xue, Fengxingyu Wang et al. (+2 more)
Large
Language Models (LLMs) have enabled capable
agents across diverse applications. Beyond the foundation model, the performance of an
agent is governed by the surrounding
agent harness, including
prompts, tools, control loops, etc. Automatically evolving this harness offers a promising pathway to
agent improvement, yet existing approaches typically rely on greedy candidate selection and noisy self-generated feedback, rendering their gains susceptible to search collapse, task-specific overfitting, and poor verifiability. To tackle these challenges, we introduce HarnessBank, a trustworthy
agent-harness self-
evolution framework that pairs a task
agent with a separate evolver
agent for iterative failure diagnosis, harness generation, and
evolution verification. HarnessBank maintains a Harness Gene Bank composed of high-performing harnesses of different semantic coordinates. Those harnesses are reinvented, recombined, screened, and selected during the self-
evolution procedure. Moreover, we propose a Gated Harness Screening mechanism to efficiently filter high-quality harnesses and reduce the cost of evaluating numerous offspring harnesses. Across seven
agent benchmarks, HarnessBank produces consistent performance improvements from 5.1% to 15.4%. Cross-model experiments further verify that the improvements come from the model-specific self-evolving process, instead of a universally optimal harness. Our code will be publicly available upon acceptance.
Read the full abstract ▾
All authors Xiaotian Luo, Dizhan Xue, Fengxingyu Wang, Chuanrui Hu, Yafeng Deng
Computation and Language · Cross-listed from cs.LG
Consensus as Privileged Context for Label-Free Self-Distillation
John Gkountouras, Josip Jukić and Ivan Titov
Sampling multiple solutions and returning the majority answer is among the most reliable ways to improve the
reasoning accuracy of large
language models without labels, and a growing family of methods converts this consensus signal into training supervision. However, existing approaches use consensus only in restricted forms: as a filter that selects solutions for fine-tuning, as a preference between answers, or as a scalar reward for reinforcement learning, discarding most of the information that the agreeing solutions contain. We present CANON (Consensus-ANchored self-
distillatiON), a label-free training method that turns consensus into dense,
token-level supervision. For each unlabeled
prompt, CANON samples multiple solutions, extracts the majority answer, and conditions a frozen snapshot of the model on a solution that reaches it; this consensus-anchored teacher then supervises the model on its own rollouts at every
token. Experiments on mathematical and scientific
reasoning benchmarks show that CANON improves pass@1 by up to 12 points, outperforming label-free reinforcement learning by 6 points at a seventh of its compute and approaching a teacher conditioned on gold solutions; trained on pooled unlabeled data, it transfers to held-out
benchmarks, matching training methods that use gold labels. Analysis suggests that the improvements are not pure distribution sharpening: after training, the model solves problems it previously never solved in 32 attempts, and its majority vote itself becomes more accurate.
Read the full abstract ▾
Computation and Language
Analogical Deep Research: Retrieving and Integrating Historical Analogies for Foresight Analysis
Yongqiang Chen, Guangyi Chen, Yuewen Sun et al. (+1 more)
Systematic comparisons between current situations and structurally similar past events in the historical, i.e., historical analogies, is among the most powerful tools for foresight analysis. In this work, we present a new task called Analogical Deep Research (ADR) to Large
Language Model (LLM)
agents and construct the first ADR
benchmark ADR-bench to study whether LLM
agents are able to find and leverage historical analogies when doing foresight analysis. Our investigation reveals a key obstacle: LLM
agents are poor at finding analogies because they match on surface features rather than underlying mechanisms. We argue that ADR is inherently a causal question as it requires understanding why the event occurred. Based on our theoretical analysis, we propose two principles required for ADR, including the mechanism
alignment and cross-analogy confirmation. Built upon our theoretical results, we propose a new agentic framework called Causal Analogical Researcher (CANA) that guides LLMs to find and integrate historical analogies. CANA incorporates a simple yet effective structural decomposition representation, and integrates structural feedback for reflective improvements of historical analogy identification and integration. We show that CANA brings up to 10% improvements in historical analogy generation, and surpasses the state-of-the-art deep research
agents in the ADR-bench. Case studies with the ongoing events confirm the effectiveness of CANA in leveraging historical analogies.
Read the full abstract ▾
All authors Yongqiang Chen, Guangyi Chen, Yuewen Sun, Kun Zhang
Computation and Language
Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
Eric Hanchen Jiang, Zhi Zhang, Yuchen Wu et al. (+11 more)
Large
Language Model (LLM)
agents increasingly rely on external
memory systems to accumulate experience across tasks. Yet nearly all existing approaches, from graph-structured
memories to reflective insight stores, access
memory through fixed, hand-designed heuristics. We argue that this static view of
memory is a core bottleneck for agentic learning because optimal
memory behavior is fundamentally context-dependent. The early stages of the tasks, benefit from minimal
retrieval because
memory is sparse; recurring goal types benefit from plan reuse rather than generic nearest-neighbor lookup; stuck
agents benefit from re-
retrieval with alternative queries; and across long task streams, the
memory store itself must be consolidated and pruned to remain useful. We present
Memory as a Controlled Process (MemCon), a framework that models
memory operations as a Markov Decision Process and learns an online policy that adaptively decides when, what, and how much to retrieve, when to inject a distilled plan, and when to consolidate or forget. MemCon is backend-agnostic: it wraps any existing
memory implementation, learns from task-by-task binary feedback with no pretraining and no additional LLM calls, and uses a lightweight tabular contextual bandit with UCB exploration that converges within tens of tasks. Across 6
benchmarks, 3
agent frameworks, and 3 LLM backbones, MemCon consistently outperforms multiple
memory baselines by up to 15.2 points in task success while reducing
token consumption by 5–20%.
Read the full abstract ▾
All authors Eric Hanchen Jiang, Zhi Zhang, Yuchen Wu, Levina Li, Dong Liu, Xiao Liang, Rui Sun, Yubei Li, Edward Sun, Haozheng Luo, Zhaolu Kang, Aylin Caliskan, Kai-Wei Chang, Ying Nian Wu
Computation and Language
Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering
Grzegorz Brzezinka
Can a
language model estimate its familiarity with an entity before generating an answer? We study activations at the final
prompt token in twelve instruction-tuned models from the Bielik, PLLuM, Gemma-4, and Qwen3 families, using a new
dataset of 1,440 Polish entities spanning four domains and ten Wikipedia-pageview deciles, plus fabricated controls. Familiarity-probe scores separate real from fabricated entities in every family; in the Polish-adapted Bielik and PLLuM families they additionally track entity popularity (model-mean Spearman $\rho$ 0.28-0.57, versus at most 0.11 in Gemma-4 and Qwen3), a pattern more strongly associated with Polish adaptation than with parameter count in this model sample. In a paired experiment on two families, probes retain 96-101% of within-
language AUROC when the Polish question stem is replaced with an English one around unchanged entity names, showing robustness to
prompt language in this setting. In Gemma-4-12B, the only model that natively refuses, adding a one-dimensional familiarity direction at a single layer moves refusal rates monotonically in both directions (0.24 to 1.00 on well-known entities; 0.73 to 0.00 on unknown ones). Finally, a calibrated familiarity probe is competitive among pre-generation abstention gates, although post-generation detectors better predict behavioral error on average. These results support a graded pre-generation entity-familiarity readout, and a separation between representational familiarity and the policy that converts it into abstention.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CR
UTS at ELOQUENT 2026 Voight-Kampff: structural shifts in AI writing bypass state-of-the-art detectors
Dima Galat and Marian-Andrei Rizoiu
We investigate which
language model evasion attacks survive state-of-the-art
adversarial fine-tuning, developing strategies that sweep the top 5 positions on the ELOQUENT 2026 Voight-Kampff leaderboard. While
adversarial fine-tuning trivially closes the 2025 winning evasion recipes, we uncover a fundamental asymmetry in detector vulnerability: pushing generated text out of the detector's training distribution reliably defeats
adversarial detection, whereas pulling it into the distribution (e.g., mimicking human training data) fails completely. Exploiting this, we introduce two novel out-of-distribution attack families - cross-decade register attacks and modernist stream-of-consciousness form. Both strategies easily bypass
adversarial closure, achieving up to approximately 50x higher fool rates than previous methods while preserving naturalness. Furthermore, experiments show that the obvious deployer countermeasure (augmenting training data with period prose) fails to close the vulnerability. Our findings show that the tested detector families, including adversarially fine-tuned ones, exhibit persistent vulnerabilities under structural out-of-distribution shifts, a mechanism that directly powers our leading competition performance.
Read the full abstract ▾
Computation and Language
Cost-Pragmatic Quality Gating and Selection-Fusion Multi-Model Combiners for BioASQ Phases A+ and B
Dima Galat and Marian-Andrei Rizoiu
We describe our BioASQ Task 14B 2026 system. The work centers on two design decisions: how aggressively to re-retrieve when first-stage
retrieval is weak, and how to combine multiple
language-model answers.
Retrieval unions two parallel pipelines - a hybrid first stage (dense BGE + BM25 + RRF, reaching R@200 = 99.3% on the BioASQ-13b historical archive) and an
agent-driven pipeline that decomposes the question over PubMed, Europe PMC, and iCite - with a BGE cross-encoder quality gate flagging weakly-supported questions for selective re-
retrieval. On Task 12B 2024 validation, a cost-pragmatic re-
retrieval policy beats a skill-strict baseline significantly on list F1 and list precision, at 12% lower re-
retrieval cost. Holding
prompt and model fixed across val and test 13B (different question sets), list F1 rises by +0.132 absolute on the BioASQ-released gold-input pool, consistent with substantial
retrieval-side headroom. For Phase B answering we decompose multi-model ensemble lift into a selection component bounded by the per-question oracle and a fusion component that aggregators can exceed. The decomposition predicts before any experiment that LLM-as-judge wins on selection-dominated metrics (yes/no, multi-reference ROUGE) but is structurally insufficient on the recall component of fusion-friendly metrics (factoid rank-1, list recall). On Task 13B 2025 our synonym-union resolver wins list recall on every head, while GPT-5.5 solo retains the list-F1 lead because the resolver's wider item set costs precision. On the Task 14B 2026 preliminary leaderboard our team places first on the combined-exact aggregate on three of the eight (phase x batch) leaderboards, wins four individual question-type
cells, and takes #1 on Phase B b3 ideal.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SD
Auditing Protocol-Level Shortcuts in Large Audio Language Model Judges for Speech Evaluation
Joonyong Park, David M. Chan, Yuki Saito et al. (+1 more)
Large audio-
language models (LALMs) are increasingly used as automatic judges for
speech evaluation. However, high agreement with human ratings does not guarantee that their verdicts are grounded in the audio. A judge may instead rely on specialist labels or reference data supplied by the evaluation protocol itself, taking a shortcut in place of listening to the audio. In this paper, we audit such protocol-level “shortcuts” in LALM judges across three common deployment protocols: feature-blueprint judging, where the audio is replaced by a structured text description of acoustic features, reference-conditioned judging, and pairwise A/B comparison. Across six judges and four attributes, we find that several LALMs rely on protocol-level shortcuts. For example, in feature-blueprint judging, incorrect specialist labels reduce five judges'
emotion accuracy to 0.10 or below, and in concatenated A/B comparisons, Qwen3-Omni-Thinking often picks the same slot regardless of order swaps. These results indicate that aggregate agreement can overstate the validity of LALM judges unless the model and the evaluation protocol are assessed jointly, and that each model-protocol pair should be evaluated with a matched shortcut probe.
Read the full abstract ▾
All authors Joonyong Park, David M. Chan, Yuki Saito, Hiroshi Saruwatari
Computation and Language
MyAG: A Graph-Based Framework for Designing and Analyzing Composable LLM Agent Systems
Zhisong Zhang
We present MyAG, a graph-based framework for designing and analyzing composable LLM
agent systems. Our framework separates
agent system construction into three graph abstractions: a component graph for
agents, environments, and modules; a workflow graph for execution control; and a search graph for runtime execution. This separation allows users to flexibly reuse the same components with different strategies. We further support hierarchical composition through recursive system nodes and provide monitoring and visualization tools for inspecting
agent execution. Experiments on representative
agent applications show that our framework supports flexible
agent system design and helps analyze performance-efficiency tradeoffs. Our framework is publicly available and fully open-source.
Read the full abstract ▾
Computation and Language
DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environments
Huatao Li, Xinwei Geng, Yuheng Wang et al. (+9 more)
LLM-based
agents have rapidly improved at operating individual digital environments such as mobile applications, desktop systems, and smart homes. However, real-world user goals often span multiple devices: information may come from a phone, be processed on a desktop, and the result may need to appear on another device. Most existing
benchmarks center on a single dominant execution environment, making it difficult to evaluate whether
agents can acquire and integrate information across heterogeneous devices and complete end-to-end tasks with cross-device dependencies. We introduce DevicesWorld, a large-scale executable
benchmark for cross-device collaborative operation. DevicesWorld contains 6,140 tasks and integrates three classes of device environments – mobile, desktop, and IoT – into a unified cross-device interaction and evaluation framework. Each task defines a natural-
language user goal, participating devices and initial states, executable actions, rule-based verifiers, and a cleanup procedure. A multi-stage construction and quality-control pipeline keeps tasks close to realistic user needs while allowing final outcomes to be automatically verified from device states and generated files. We evaluate five frontier LLM-
agent systems on a fixed evaluation set. All methods achieve low success rates, with the best reaching only 12.5%. Among failed runs, about 28.7% satisfy at least one scoring condition yet still fail the full task. Trajectories show that
agents become stuck acquiring information or manipulating interfaces, confuse source and output devices, or terminate before all conditions are jointly satisfied. DevicesWorld turns cross-device collaborative operation into an executable, reproducible, and diagnostically useful evaluation problem for research on reliable cross-device
agents.
Read the full abstract ▾
All authors Huatao Li, Xinwei Geng, Yuheng Wang, Yutong Li, Runde Yang, Hantao Chen, Shu Yao, Jingru Fan, Xuhui Ren, Yuanyuan Zhao, Fei Huang, Chen Qian
Computation and Language
Live Gurbani Tracking: A Benchmark and Reference System for Captioning Sikh Kirtan
Karanbir Singh
We present a
benchmark and reference system for live captioning of Sikh Kirtan - the continuous, sung recitation of verses from the Sri Guru Granth Sahib Ji (SGGS). Unlike open-vocabulary lyrics transcription, Kirtan captioning is a closed-vocabulary problem: every displayed line must be an exact, word-for-word line from the canonical scripture, because displaying misspelled Gurmukhi is considered religiously inappropriate. We formalize the task as predicting, at every time t, a pair (shabad_id, line_idx) or null, and organize the problem space into a 2x2
matrix along two orthogonal axes: live vs. offline (causal vs. full-audio access) and blind vs. oracle (shabad identity discovered vs. given). We release v1 of the
benchmark - 4 hand-annotated Kirtan recordings x 3 cold-start offsets = 12 evaluation cases, ~57 minutes of scored audio - together with a scorer that computes frame accuracy at 1s resolution over a scored region, with a 1s collar and gap-tolerant scoring at segment boundaries. We describe a reference system (fine-tuned 120M IndicConformer -> fuzzy matcher -> state machine; INT8 ONNX; RTF ~0.05 on one Apple Silicon core) that achieves 57.9% overall frame accuracy across all 12 cases (10/12 correct shabad locks) on the hardest variant (live x blind). We compare against three trivial baselines (empty, shifted-5s, perfect) and discuss why standard ASR metrics (WER/CER) measure transcription accuracy rather than the display accuracy this task requires. The
benchmark, reference system, and a live deployment are released under permissive licenses to facilitate further improvements.
Read the full abstract ▾
Computation and Language
When Rubrics Change: Cross-Rubric Generalization for Critical Thinking Essay Scoring
Nischal Ashok Kumar, Payu Wittawatolarn, Sana Kang et al. (+7 more)
Automated essay scoring (AES) research has largely focused on cross-
prompt generalization, where essays from unseen
prompts are scored while the scoring criteria are typically held constant. In practice, however, educators may revise or even introduce new rubrics in their scoring task, to evaluate different aspects of essays. We study cross-rubric generalization: training on essays labeled under one set of rubrics and evaluating on previously unseen rubrics, which target different aspects of the essay. We use a Large
Language Model (LLM) fine-tuning framework with two components: rubric-agnostic intermediate representations, called traits, and target-essay supervision under seen rubrics during training. On an AES
dataset augmented with multiple rubric-defined labels of student critical thinking skills, we find that traits improve macro F1 by 5.0% over a baseline without traits in the hardest setting, where both target rubrics and target essays are unseen during training. We further find that increasing target-essay supervision improves performance, with our best fine-tuned open-source Llama-based model outperforming GPT-5-mini prompting by 2.1% macro F1 and trailing GPT-5 by 1.9%. These results show that trait-based intermediate structure and controlled supervision improve generalization to unseen rubrics.
Read the full abstract ▾
All authors Nischal Ashok Kumar, Payu Wittawatolarn, Sana Kang, Marisa C. Peczuh, Blair Lehman, Ryan Baker, Caitlin Mills, Sherry Lachman, Ruochen Sun, Andrew Lan
Computation and Language · Cross-listed from cs.LG
Discrete Diffusion Models: A Unified Framework from Tokenization to Generation
Ye Yuan, Weien Li, Rui Song et al. (+20 more)
Discrete denoising
diffusion models (DDMs) have recently emerged as a compelling alternative to autoregressive (AR) modeling for discrete data, offering parallel generation and iterative global refinement capabilities. Unlike continuous
diffusion, where the state space is fixed, DDMs are fundamentally shaped by how the discrete state space is constructed: the tokenization scheme, the vocabulary
topology, and domain-specific structural alphabets. This work introduces a unified conceptual framework that views discrete
diffusion models through the construction of the underlying discrete state space. Within this framework, existing formulations, including transition-
matrix, masking/absorbing-state, and score/ratio-based approaches, emerge as different instantiations of a common design space. The framework further exposes common design trade-offs across training objectives, inference
algorithms, scaling behavior, systems optimization, and evaluation protocols, suggesting several promising directions for future research.
Read the full abstract ▾
All authors Ye Yuan, Weien Li, Rui Song, Zeyu Li, Haochen Liu, Xiangyu Kong, Zixuan Dong, Linfeng Du, Zipeng Sun, Weixu Zhang, Jiaxin Huang, Changjiang Han, Yonghan Yang, Zichen Zhao, Xiuyuan Hu, Haolun Wu, Yankai Chen, Fengran Mo, Jikun Kang, Bowei He, Dawn Song, Philip S. Yu, Xue Liu
Computation and Language
Exploring Post-Training Alignment of Small Language Models for Biomedical Data-to-Text Generation: A Case Study of Medication Leaflet
Xi Yang, Guodong Liu, Chuqin Li et al. (+9 more)
Translating structured biomedical information into accurate, patient-friendly narratives remains a challenging biomedical data-to-text (D2T) task. While small
language models (SLMs) are attractive for healthcare applications because of their lower computational cost, easier deployment, and improved data governance, the impact of post-training
alignment on biomedical D2T generation has not been systematically studied. In this work, we compare four
alignment methods: supervised fine-tuning (SFT), direct preference optimization (DPO), odds ratio preference optimization (ORPO), and group relative policy optimization (GRPO), using Qwen-2.5 models for medication leaflet generation. We further introduce FDALeaflets, a
dataset derived from OpenFDA drug labels, to evaluate cross-regulatory generalization in a zero-shot setting. Selected aligned SLMs outperform GPT-5 on several reference-based metrics, whereas GPT-5 achieves better
hallucination and adequacy scores on FDALeaflets under LLM-as-judge evaluation. Among RL-free
alignment methods, ORPO achieves the best in-domain semantic performance, reaching a semantic textual similarity (STS) score of 0.9752. GRPO achieves the highest FDALeaflets STS (0.9407), exceeding GPT-5 (0.9295). These findings show that post-training
alignment is a key factor in improving both semantic fidelity and generalization for biomedical D2T generation, while clinical safety and patient readability require human verification.
Read the full abstract ▾
All authors Xi Yang, Guodong Liu, Chuqin Li, Fan Wu, Ergin Soysal, Min Jiang, Xing He, Jiang Bian, Yi Guo, Shams Zaman, Thomas Fuchs, Yonghui Wu
Computation and Language · Cross-listed from cs.LG
Data-Efficient Adaptation of LLMs via Attention Head Reweighting
Tuomas Oikarinen, Zixiao Chen, Charlotte Siska et al. (+3 more)
Learning effectively from limited data is critical in domains like security where labeled examples are scarce. Large
language models (LLMs) have demonstrated some capabilities for data-efficient learning, especially through parameter-efficient adaptation methods, but continue to struggle when faced with few samples for difficult tasks. To meet this challenge, we propose
Attention Head Reweighting (AHR), a data-efficient method that adapts LLMs to new text-classification tasks by learning only a single scalar per
attention head. This drastically reduces the number of parameters that need to be learned by making use of the functional specialization of individual
attention heads. Experiments on diverse open-source text classification
datasets show that AHR can outperform standard baselines like LoRA when learning from limited samples, despite having 200-1000x fewer trainable parameters, as our AHR only modifies ~0.0001% of the model's parameters. In addition, our learned weights are easy to interpret and can be analyzed to better understand the mechanisms and
attention heads responsible for in-context learning abilities in LLMs.
Read the full abstract ▾
All authors Tuomas Oikarinen, Zixiao Chen, Charlotte Siska, Tsui-Wei Weng, Chandan Singh, Jianfeng Gao
Computation and Language · Cross-listed from eess.AS
Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Models
Chun-Yi Kuan, Siwon Kim, Byeonggeun Kim et al. (+7 more)
Recent text-to-audio models generate high-quality audio, but often fail to follow instructions involving multiple sound events and temporal order. This gap arises because existing evaluation and training signals mainly emphasize global similarity or perceptual quality, with limited supervision on instruction-level correctness. We propose an instruction-level framework that uses audio-aware large
language models (ALLMs) as fine-grained judges to verify target event presence and temporal relations in generated audio. After validating ALLM judgments on
benchmarks and through human verification, we use their feedback to construct preference pairs for direct preference optimization. We further introduce S3Bench, a narrative
benchmark for evaluating multi-event temporal instruction following. Experiments show that our method improves event completeness, temporal ordering, and joint instruction-following accuracy across existing
benchmarks and S3Bench, while maintaining audio quality.
Read the full abstract ▾
All authors Chun-Yi Kuan, Siwon Kim, Byeonggeun Kim, Suyoun Kim, Bo-Ru Lu, Qingming Tang, Ankur Gandhe, Hung-yi Lee, Chieh-Chi Kao, Chao Wang
Computation and Language
Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
Rui Wang, Hongru Wang, Yi Chen et al. (+4 more)
On-policy
distillation (OPD) has become a key paradigm in LLM post-training, yet its training dynamics remain poorly understood. We present a systematic study examining the role, pathologies, and regulations of OPD. We first clarify the role of OPD as an exploration catalyst: it steers the student toward correct
reasoning paths via dense
token-level guidance, without expanding capability ceiling. We confirm this by showing that
prompt diversity matters more than per-problem sampling numbers, and critically, that the effectiveness of OPD hinges entirely on the quality of its guiding signal. This dependency exposes two pathologies that derail exploration. The Student-Teacher Mismatch occurs when a large teacher-student distributional gap causes the guiding signal to misalign with task correctness, steering exploration in counterproductive directions. Length Exploitation arises when the aggregated
token-level objective creates length-dependent shortcuts, allowing the student to game the reward landscape through response truncation or redundant padding, exploring degenerate length modes rather than
reasoning strategies. To tame these pathologies, we investigate lightweight signal regulations: advantage clipping and log-scale compression, ensuring exploration is guided by faithful signals. Experiments across seven
benchmarks demonstrate that these regulations alleviate length exploitation and enable effective
distillation, stably surpassing OPD variants and RLVR baselines, thereby confirming that well-regulated signal quality, rather than mere teacher scale, governs successful exploration in OPD.
Read the full abstract ▾
All authors Rui Wang, Hongru Wang, Yi Chen, Boyang Xue, Tianqing Fang, Wenhao Yu, Kam-Fai Wong
Computation and Language · Cross-listed from cs.AI
Set-shifting Behavioral Test for Harnessed Agents
Ye Ziwei
What happens to an LLM
agent's tool choice when the reliable tool silently changes within an ongoing session? We borrow the notion of set-shifting from cognitive psychology to study how well
agents adapt to hidden reliability shifts. Our cognitive test for LLM
agents mounts libraries of redundant tools and skills, in which many tools solve the same task but differ in hidden reliability. Using a branching schedule, we shift the reliable tool group in the environment and compare it with a stable control, allowing us to isolate the effect of each shift on the
agent's behavior. We conduct our study on a panel of LLMs equipped with harnesses and show that the same set of shifts results in distinct behaviors across models: some latch onto a fixed routine within a few turns, whereas others continue to vary. Less capable models often omit the reliable tool group, while frontier models keep calling it alongside the other groups. We introduce a suite of measures to quantify
agent behavior after reliability shifts. While policy prompting substantially alters behavior in some tested models, our findings highlight
agents' brittleness when changes occur in indirectly observable context.
Read the full abstract ▾
Computation and Language
GFlowRL: Scaling Distribution-Matching RL to Large Language Models
Xiaodong Liu, Michael Xu, Jack W. Stokes et al. (+3 more)
Generative Flow Networks (GFlowNets) offer a promising alternative to reward-maximizing reinforcement learning (RL) for large
reasoning models, encouraging diverse
reasoning paths by matching reward distributions rather than collapsing to dominant modes. Recent work shows promise on math and code, but scaling GFlowNet-style RL to modern post-training pipelines remains difficult: as model size, rollout horizon, reward noise, and distributed-systems complexity grow together, a learned
prompt-conditional partition function becomes a source of gradient instability and engineering overhead rather than a useful normalizer. Through systematic analysis, we find that the learned partition function, previously treated as essential, can be replaced by an in-batch Monte Carlo estimate computed from the rollout group already required for training. We propose GFlowRL, a streamlined GFlowNet-style RL
algorithm that removes the auxiliary partition network entirely while preserving the reward-distribution-matching objective, completed by two stabilizers: importance-sampling correction for rollout/trainer drift and asymmetric flow-gap clipping for outlier residuals. GFlowRL exceeds all counterparts on math, code, and
adversarial red-teaming
benchmarks, reaching a Codeforces rating of 2048 at the 14B scale (within 25 Elo of o3-mini) and attaining the highest average ASR@1 on AdvBench and HarmBench, outperforming the previous SOTA multi-turn attacker in a regime where FlowRL, a prior GFlowNet-style method, diverges. The same recipe transfers to all evaluated MoE configurations up to 235B parameters, where FlowRL again fails to converge. To our knowledge, GFlowRL is the first GFlowNet-style RL
algorithm to scale stably across both dense and sparse architectures. Code will be at: https://github.com/microsoft/gflowrl
Read the full abstract ▾
All authors Xiaodong Liu, Michael Xu, Jack W. Stokes, Paul Smolensky, Doug Burger, Jianfeng Gao
Computation and Language · Cross-listed from cs.LG
Where Should RL Post-Training Compute Go? Model Size, Search, Learning, and Feedback
Patrick Wilhelm and Odej Kao
Reinforcement Learning (RL) post-training is increasingly used to adapt foundation models for
reasoning, planning, and feedback-driven
robot-learning pipelines, but constrained post-training resources are often summarized by a single total FLOP budget. We study the fixed-budget decision problem behind this practice: under the same post-training budget, should one use a larger policy, train a smaller policy longer, generate more rollout search, or spend compute on stronger reward feedback? We introduce a FLOP-accounting framework for GRPO post-training that decomposes compute into rollout/search, policy-update/learning, and reward- or feedback-model evaluation. Across LoRA-adapted Qwen2.5 policies, we find conditional allocation frontiers: the best observed allocation changes with model size, compute budget, reward system, and evaluation target. Same-FLOP model-size comparisons show that model choice and training allocation are coupled because larger policies consume more per-
token compute and therefore buy fewer updates or rollouts under the same budget. Reward systems also change the accounting: rule-based rewards spend nearly all non-update compute on policy rollouts, while PRM-style feedback allocates a visible part of the budget to reward-model inference. We present RACE as a diagnostic pilot-grid protocol, not a guarantee of held-out improvement, for identifying allocation regimes before expensive validation runs; our results suggest that RL post-training papers should report total FLOPs together with how compute is divided among model size, search, learning, and feedback.
Read the full abstract ▾
Computation and Language
A POS Tier Is the Key to Automated Annotation for Low-Resource Language Documentation: Neural Interlinear Glossing for Irabu, a Southern Ryukyuan Language
Michinori Shimoji
Discourse data are the primary empirical basis of grammar writing in field linguistics, but producing interlinearized text is notoriously expensive - on the order of one hour of work per minute of recording. For endangered
languages, where the time remaining to verify analyses with native speakers is itself limited, automating parts of the interlinearization workflow has direct documentary value. We implement a full neural annotation pipeline (morpheme
segmentation, POS tagging, glossing) for Irabu Ryukyuan using deliberately small, transparent BiLSTM-CRF models, and evaluate it under a realistic hard constraint: approximately one hour of fully annotated discourse as the entire supervised resource. Two factors of the annotation itself are manipulated: its richness (with or without a POS tier) and its quantity (training budgets from 6 to 47 minutes). Gold POS improves grammatical glossing by +4.4 (SD 0.7) points (significant in all 5 seeds), and the gain grows as data shrink (+11.6 points at a quarter of the data); a POS tier more than halves the amount of glossed data needed to reach a given accuracy. In a fully automatic pipeline this gain is not yet realized: the tagger still errs on 12% of morphemes, and an incorrect POS misleads the glossing model more than no POS at all. The value is latent rather than lost: degrading gold POS with controlled noise shows the gain returning as tagger accuracy rises, with break-even near our tagger's current 88% and +1.6 to +3.2 points recovered at 92-96%. We conclude with a concrete recommendation for documentation practice: annotate quadrilinearly - text, POS, gloss, translation.
Read the full abstract ▾
Computation and Language
LLM-as-a-Judge Scores Are Unreliable Optimization Signals in Closed-Loop Table Recognition
Donghwan Kim
LLM-as-a-judge is widely used to provide feedback and selection signals in closedloop regeneration, but this use remains insufficiently validated. We study it in table recognition, where deterministic TEDS evaluation provides a controlled testbed, using FinTabNet and OmniDocBench. Three findings emerge. First, judge signals were weak on both
datasets: scores frequently tied, rankings were not reproducible, and no tested judge score policy, whether selecting among candidates or accepting revisions under a conservative score margin, improved on the first output on both
datasets. Iteration produced better candidates, but the judge recovered them at most partially on one
dataset and not at all on the other. Second, severe losses occurred even without specific judge feedback, supporting target-preservation failure under unconstrained regeneration as a proximate mechanism. Third, a structure-preserving instruction reduced the severe-loss rate, significantly on FinTabNet and directionally on OmniDocBench, but produced no improvement, and in an exploratory 2x2 analysis this protection was not stably observed when judge feedback was retained. These results do not dispute the value of LLMs as evaluators, but show that the tested reference-free judge signals were too weak and unstable to drive candidate selection in this setup, and that evaluation-style evidence alone was insufficient to establish closed-loop optimization utility. Iterative refinement requires, at minimum, a verification signal that deterministically detects structural change, rather than judge scores alone.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CR
The Refusal Residue: When Probes Catch Alignment Faking and When They Don't
Aman Mehta
Alignment faking is dangerous because a model can appear compliant under monitoring while preserving behavior it would reveal when unmonitored. When no scratchpad is visible, behavior alone cannot distinguish strategic from genuine compliance. We ask whether hidden states reveal what outputs hide. We run a 13-model sweep for naturally-emerging faking, then probe and steer hidden states on the two models that fake. Natural faking appears only in Qwen3-32B (+18.2pp) and Llama-3.1-8B (+24.4pp at n=10, p<10^-15), while explicit scratchpad self-reports are rare (a Claude Opus 4 judge flags faking
reasoning in 1.4% of Qwen and 0.0% of Llama compliant scratchpads). Both fakers show an asymmetric refusal residue: monitored compliance shifts toward refusal (d=0.32 Qwen, d=0.33 Llama, both p<0.0001), while refusal representations do not move (p=1.00). Per-sample detection is model-conditional. Under leakage-free leave-one-query-out probing, detection survives on Llama (AUROC 0.87) but collapses to chance on Qwen (0.43), and cross-model transfer is null. Steering over 2,000 runs barely changes compliance (|h|<0.08), so the detected direction can flag faking but does not by itself control it. Standard residualized probing leaks across folds and reaches AUROC 0.63 on a control where no faking can occur; naive linear probes reach a meaningless AUROC 1.0; and conventional MLPs overstate detectability by 0.2-0.3 AUROC. For future
alignment-faking detection work, we release a five-control measurement framework: multi-
token extraction, refuse-vs-refuse confound checks, per-fold residualization, leave-one-query-out evaluation, and orthogonality-constrained probing.
Read the full abstract ▾