Computation and Language
MILO: Efficient Many-shot In-Context Learning with Block-wise Low-rank Compression
Youpeng Zhao, Tian Tan, Liqian Peng et al. (+2 more)
Many-shot in-context learning (ICL) enables large
language models (LLMs) to adapt to complex tasks by conditioning on thousands of demonstration examples, but this paradigm shifts the inference efficiency bottleneck to the key-value (KV) cache
memory. Due to the linear scaling behavior of the KV cache, storing these intermediate tensors has become a paramount challenge for both online serving and on-device deployment. To address this issue, we propose a novel compression framework, termed MILO, that exploits the low-rank redundancy inherent in many-shot contexts. Specifically, MILO features a block-wise low-rank compression strategy that compresses the KV cache at the block granularity, where each block contains multiple many-shot examples. Furthermore, to handle the heterogeneous context density across different blocks, MILO dynamically allocates rank budgets based on the information
entropy, preserving the fidelity of critical blocks while aggressively compressing redundant ones. Experimental results on Qwen2.5 models demonstrate that our method achieves up to 50% reduction in KV cache
memory and 1.8x throughput improvement, with negligible performance degradation on classification and
reasoning benchmarks, significantly outperforming prior baselines.
Read the full abstract ▾
All authors Youpeng Zhao, Tian Tan, Liqian Peng, Jun Wang, Alec Go
Computation and Language
Multi-Task Learning by using Contextualized Word Representations for Syntactic Parsing of a Morphologically Rich Language
Toqeer Ehsan, Miriam Butt, Sarmad Hussain et al. (+2 more)
We address the challenge of syntactic parsing for Urdu, a morphologically rich
language, and present state-of-the-art results for both constituency and dependency parsing. This paper offers four major contributions: 1) the conversion of the CLE-UTB phrase structure treebank into a dependency treebank by developing
language-specific head-word and phrase-to-dependency label mapping rules; 2) a novel sequence labeling scheme that transforms the parsing task into a unified representation; 3) the training of contextualized word representations on a large 220 million
tokens Urdu corpus collected from the web; and 4) development of parsing framework using two learning paradigms, single-task and multi-task learning. Several post-processing rules are applied to improve the quality of the automatically converted dependency structure treebank. The proposed sequence labeling scheme enables the use of a shared architecture that learns the syntactic structures from both grammatical structures simultaneously and hence improves generalization. Experiments show that the multi-task learning setup significantly enhances parsing performance, achieving an F1 score of 91.39 for constituency parsing (an improvement of 3.29 points) and a labeled attachment score of 85.69 for dependency parsing (an improvement of 1.49 points). These results demonstrate that learning cross-task representations provides measurable benefits and advances the state of syntactic parsing for Urdu.
Read the full abstract ▾
All authors Toqeer Ehsan, Miriam Butt, Sarmad Hussain, Hassan Alhuzali, Ali Al-Laith
Computation and Language
Encoded but Not Decoded: Layer-Localized Evidence for a Three-Level Gap in LLM Syntax
Zhenyan Lu, He Wang and Xiaohui Huang
A
language model can fail a syntactic test in two distinct ways: by not encoding the relevant structure, or by encoding it but failing to use it at the output. Behavioral evaluation alone cannot tell these apart. We propose a three-level evaluation framework (behavioral deployment, LM-head readout, and probe recoverability) measured on the same items under the same binary decision. Using a compact trilingual (English, Chinese, German) control-dependency
benchmark, we find that probe recoverability exceeds or equals LM-head readout, which in turn exceeds or equals behavioral deployment, across seven models and all three
languages in the aggregate. The recoverability surplus is never negative across all 14 (model, task) conditions. The disconnect concentrates in subject-control, where a nearest-noun heuristic gives the wrong answer. The single largest gap (0.653) appears on Qwen3-0.6B Instruct in question answering. The gap persists at Qwen3-14B Instruct. Instruction tuning degrades deployment more than encoding in percentage terms. We rule out option-position bias, late-layer erasure, output-formatting artifacts, and probe-training variance. The pattern is consistent with decoding that favors surface shortcuts, and the behavior-probe gap measures the strength of that preference. Activation patching shows the gap is layer-localized. Under instruction tuning, the LM-head-decoded layer shifts approximately ten layers later than the probe-decoded layer. These findings argue that behavioral evaluation understates what models encode, while probing alone overstates what they deploy.
Read the full abstract ▾
Computation and Language
Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs
Pavel Tikhonov, Anton Korznikov, Matvey Mikhalchuk et al. (+6 more)
While Large
Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams are linearly combined, the model outputs a superposition of the individual next-
token distributions. We term this the Superposition Linearity Hypothesis. We provide evidence that superposition is an intrinsic property of the
Transformer architecture rather than an emergent consequence of training; in fact, we observe that it tends to diminish as pretraining progresses. However, we demonstrate that linearity can be substantially restored through lightweight fine-tuning, significantly reducing the divergence between the predicted next-
token distribution and the average of the individual next-
token distributions. Finally, we introduce a guided decoding procedure that disentangles superposed outputs, enabling the simultaneous generation of two coherent continuations from a single forward pass.
Read the full abstract ▾
All authors Pavel Tikhonov, Anton Korznikov, Matvey Mikhalchuk, Nikita Dragunov, Temurbek Rahmatullaev, Polina Druzhinina, Anton Razzhigaev, Ivan Oseledets, Elena Tutubalina
Computation and Language · Cross-listed from cs.AI
PUBG Ally: A Conversational Embodied Agent as an AI Teammate
PUBG Ally Team: Irene Chen, Youngin Cho, Seungjun Chung et al. (+21 more)
We introduce PUBG Ally, an embodied
agent for PUBG: BATTLEGROUNDS that can reason, act autonomously, and play alongside players as a voice-enabled teammate. Building such a teammate requires combining two difficult capabilities: it must perceive and respond to a constantly changing game world under strict latency constraints while interacting naturally with players, keeping its
speech synchronized with its actions. Ally therefore combines agentic tool use with real-time game control. A
language-model
agent uses a controlled interface to inspect game information, interpret player
speech, maintain context, decide what to say, and issue high-level action choices that steer a faster control layer for movement, combat, and recovery. Because the player's and Ally's
speech and actions continually shape each other and the course of the match, training requires data from actual gameplay. We therefore collect data across nearly 39k sessions in which real players play alongside Ally, recording gameplay, player
speech,
agent decisions, tool use, actions, and player feedback, and use these records for iterative training. To evaluate teammate quality, we use player feedback and preference comparisons to identify gaps between offline evaluations and player preferences, and iteratively refine the evaluation criteria. Deploying Ally in live service further requires low-latency on-device execution and safeguards for player-facing communication, which we address through model compression, context compaction, targeted safety training, runtime guardrails, and
memory redaction. During the live service, we surveyed players in 141 countries. Among respondents whose play with Ally was confirmed in game records, positive responses exceeded negative responses by 25.1 percentage points when asked whether they would recommend Ally, with players describing Ally not only as a tool but also as a teammate or companion.
Read the full abstract ▾
All authors PUBG Ally Team: Irene Chen, Youngin Cho, Seungjun Chung, Jimin Hong, Hyeonbin Hwang, Hyeojung Im, Insub Im, Jaeseung Jeon, Seohyeon Jung, Beomsoo Kim, Byeongju Kim, Dohyun Kim, Dongwon Kim, Eunchong Kim, Hongmin Kim, Hyeonghwan Kim, Hyunseung Kim, Sungwoo Kim, Kangwook Lee, Minkyoung Park, Sue Hyun Park, Hyoseok Seol, Yujeong Son, Kiyoon Yoo
Computation and Language
ChunkRank: Model-Aware Text Chunking and Abstention-Aware Answer Selection for LLM Pipelines
Amit Nautiyal, Ayush Bhatt and Gaurav Nautiyal
We present ChunkRank, an open-source Python library that derives chunk boundaries from a target model's tokenizer and context window, and selects an answer among candidates produced independently per chunk. It ships a validated registry of 90 models across 15 providers and six answer-selection methods, and needs only three core dependencies. For chunking, ChunkRank avoids context-window overflow automatically from the model name, whereas character-based splitters overflow or waste the budget, and a fidelity study across 11
languages shows why
token-exact budgets matter beyond English. For answer selection we report a negative result: on NaturalQuestions, TriviaQA and HotpotQA, with extractive and generative readers, no content-based ranker reliably beats taking the first non-empty answer. The reason is reader abstention on chunks that lack the answer, not answer position. A long-context baseline shows that chunking matches single-call reading on single-hop questions, so ChunkRank targets small-window and beyond-window settings. Code, registry and evaluation harness are released.
Read the full abstract ▾
Computation and Language
CORDIAL: Calibrating Ordinal LLM Outputs from Few Labels
Xiangwei Wang, Peng Wang and Saman Halgamuge
A large
language model (LLM) can turn a text into a distribution over an ordered scale, but that distribution is a noisy measurement: saturated, compressed or exaggerated, and biased in a consistent direction. We propose CORDIAL, which treats the model's output as a noisy reading of the true label and corrects it with a channel of five interpretable parameters. The channel is small enough for its posterior to be averaged from a handful of labels, and we prove that the resulting calibration preserves first-order
stochastic order. On Amazon reviews and CMU-MOSEI transcripts with four LLMs, CORDIAL has the lowest log loss among nine calibrators in 76 of 80 settings with 5 to 100 labels; with 20 labels and the main 7B reader, it matches the strongest baseline using 28-54 labels. The same posterior lets us learn priors from other tasks and fuse several LLMs. Unrestricted calibrators such as Dirichlet calibration overtake it only as the calibration set grows into the hundreds or thousands.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Learning to Ideate for Scientific Impact
Shubham Kale, Aniketh Garikaparthi and Manasi Patwardhan
Scientific ideation is increasingly mediated by large
language models, but current ideation systems are usually trained and evaluated on immediately judgeable proxies such as novelty, clarity, and feasibility. This leaves open whether delayed signals of scientific uptake can be used as feedback for steering models toward research directions with higher expected impact. We study this question using citation-normalized impact as a noisy but scalable proxy for scholarly uptake. We construct a large-scale
dataset from over 100K computer science papers by extracting goal-conditioned idea descriptions and assigning each paper an ordinal, year-normalized citation label. We then train a goal-conditioned reward model to predict citation-impact labels from research goal and idea pairs, and use this reward to align an idea generator through supervised fine-tuning followed by reinforcement learning. To reduce circularity, we evaluate generated ideas with a held-out, reference-grounded protocol that compares model outputs against historical ideas under the same research goal and weights judgments by the reference idea's citation-impact label. Experiments show that our RL-tuned model consistently produces ideas with higher estimated impact than both the base model and supervised fine-tuning baselines. Our findings position scientific impact as a practical, outcome-grounded feedback signal for aligning LLMs in open-ended scientific discovery.
Read the full abstract ▾
Computation and Language
Adaptive Fisher-Whitened Cross-Covariance for Low-Resource Speech Recognition
Asmee Mishra, Mengjie Qian, Brechtje Post et al. (+1 more)
Adapting multilingual
speech foundation models to low-resource
languages remains difficult, especially for
languages that are poorly represented during pre-training. While parameter-efficient fine-tuning (PEFT) reduces the cost of adapting large models, conventional approaches such as LoRA rely on generic low-rank parameterizations and do not explicitly use downstream task information to define the adaptation subspace. To investigate whether task-informed PEFT can better support low-resource ASR, we apply Fisher-Whitened Cross-Covariance Analysis (FCCA) to Whisper and Qwen3-ASR, and introduce two complementary extensions: Asymmetric-Coupled FCCA (AC-FCCA), which exploits structured cross-layer sharing, and Adaptive-Rank FCCA (AR-FCCA), which reallocates adaptation capacity across projection
matrices under a fixed parameter budget. Under controlled multilingual experiments, we evaluate these approaches on
languages that are poorly represented or unsupported during pre-training alongside well-represented
languages. Standard FCCA is competitive with, and usually outperforms, trainable-parameter-budget-matched LoRA. AR-FCCA provides the most consistent improvement over standard FCCA across both model architectures, with statistically significant gains in several evaluation settings, while retaining the same number of trainable parameters. These results show that task-informed subspace construction can be effective for low-resource
speech adaptation, and that adaptive rank allocation provides a robust way to improve parameter efficiency without increasing model capacity.
Read the full abstract ▾
All authors Asmee Mishra, Mengjie Qian, Brechtje Post, Kate Knill
Computation and Language
Benchmarking and Domain Adaptation of Automatic Speech Recognition (ASR) for Adolescent Health Communication in Ghanaian Languages
Stephen E. Moore, Akwasi Asare, Mich-Seth Owusu et al. (+3 more)
This paper presents an end-to-end study of automatic
speech recognition (ASR) for adolescent health communication in three Ghanaian
languages (Twi, Dagbani, and Ewe). The work proceeds in three connected stages; First, we
benchmark five ASR systems (three
language-specific Wav2Vec2 models and two
multimodal LLMs, Gemma 3n and Gemma 4) on a general-domain Bible corpus and a Youth Adolescent Sexual and Reproductive Health (ASRH) Domain ASR
dataset, using Character and Word Error Rate (CER, WER). Second, guided by the
benchmark, we perform supervised domain adaptation: although Gemma 4 was the strongest zero-shot candidate, fine-tuning it proved computationally infeasible, so we pivoted to the compact Qwen3-ASR-0.6B, fine-tuned on a large Ghana Bible corpus (~90k samples) and evaluated strictly on held-out human-collected in-domain audio. Fine-tuning reduced WER on every
language, most dramatically for Ewe (WER from 109.3% to 64.8%, a drop of 44.5 pp; CER from 65.1% to 24.9%). Third, we validate the work through KasaHealth, a live voice-first ASRH application deployed in all three
languages, complemented by Senti-Check, a technical evaluation harness. KasaHealth was tested by 50 community respondents and achieved a 100% chat-approval rate, a 72% Good-or-Excellent translation rating, and a 92% would-recommend rate, while surfacing the domain gaps that most constrain real-world use. Across all three stages the evidence converges: for these
languages the binding constraint is validated in-domain data, not model capability or computation.
Read the full abstract ▾
All authors Stephen E. Moore, Akwasi Asare, Mich-Seth Owusu, Paul Azunre, Joel Budu, Lawrence A. Adu-Gyamfi
Computation and Language
TimeBraid: Unifying Time Series and Language for Understanding and Forecasting
Xinyue Wang, Jiacheng Pang, Kun Zhou et al. (+7 more)
We present TimeBraid, a series of unified time-series and
language models that align pretrained
language models and pretrained time-series foundation models through interleaved global residual
attention layers. Each model inherits knowledge, instruction following, and
reasoning from one side, continuous-signal perception and zero-shot forecasting from the other, and fuses the two in a shared representation space where both modalities are understood and generated. We study the design choices that make such unified modeling work: where to align the two representation spaces, how to ground
language in temporal structure, how to balance understanding with generation, and how to keep joint optimization stable. The resulting recipe combines a unified prompting scheme for diverse time-series and text tasks, stabilized joint training, and supervision from 2.2M curated series–text pairs and 4.9M instruction-tuning samples. Across
benchmarks spanning time-series perception, understanding,
reasoning, and both context-aided and unimodal forecasting, TimeBraid remains competitive with far larger general-purpose models and task-specific counterparts.
Read the full abstract ▾
All authors Xinyue Wang, Jiacheng Pang, Kun Zhou, Kexin Zhang, Defu Cao, Fan Feng, Faisal, Songyao Jin, Yan Liu, Biwei Huang
Computation and Language
Decision Models vs. LLMs as Rubric Judges: Cheaper, Faster, and Wrong in the Same Places
Delip Rao and Chris Callison-Burch
LLM judges now score outputs against rubrics, in
benchmarks and as training rewards. Decision models return probabilities over permitted answers with a confidence score but no free text. Two such models are Jev and an open reproduction, Perplexity's Decider. We compare each with three flash-tier LLM judges on nine panels from seven
benchmarks with human judgments, giving every judge identical criterion texts. The LLM judges run in two setups, reading a whole rubric at once or one criterion at a time. Our findings suggest both decision models can often stand in for them. The LLM judges cost 16 to 357 times as much and take 5 to 350 times as long, yet in both setups each decision model trails them significantly in at most 6 of 27 comparisons. The two kinds of judge err alike, agreeing more with one another than with the human-annotated judgment labels on ordinal criteria. Over 85% of flash-tier LLM verdicts on either decision model's most confident errors repeat them, where independent errors would give about half. Each decision model's confidence ranks its errors on most ordinal panels, so each is a natural first stage for a cascade deferring uncertain verdicts to an LLM judge. Yet with flash-tier fallbacks, such cascades gained a mere 2.7 points at best over the strongest single judge, even with oracle thresholds, suggesting they mainly save cost. A cascade needs a fallback that is right where its first stage errs, but the LLM judges are mostly wrong in the same places as Jev, and the Decider is already the strongest single judge on most panels. Even each vendor's higher-tier model repeats those errors, and as a fallback it lifted no cascade more than 1.7 points above both tiers' strongest single judge. Holding across setups, effort levels, and tiers, these findings suggest designing future decision models afresh to get right what LLM judges get wrong, and vice versa.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
C3M: Cross-Session Multimodal Memory Maintenance for Long-Horizon Tasks
Xueshu Chen, Yan Wang, Zihao Xue et al. (+5 more)
Long-horizon tasks require preserving and later recovering cross-session evidence under a bounded, query-blind
memory budget. Existing compression can discard fine-grained visual cues or conflate semantically similar but incompatible observations. We present C3M, a cross-session
multimodal memory organization that maintains a bounded active index over persistent source text-image evidence. Relation-aware updates consolidate safe redundancy while preserving complementary and incompatible records. At query time, budgeted routing selects useful index pages and expands their associated source evidence under a fixed reader budget. Together, these mechanisms establish a compact, provenance-preserving
multimodal memory organization for cross-session long-horizon tasks, retaining temporal distinctions and source links required for reliable downstream
reasoning. Code is available at https://github.com/HuzhouNLP/C3M.
Read the full abstract ▾
All authors Xueshu Chen, Yan Wang, Zihao Xue, Jiefu Li, Zhenfang Liu, Jayden Chen, Zhen Bi, Jungang Lou
Computation and Language
TTLab at StanceEval-2026: A Cloze-Style Prompting Approach for Arabic-Language Stance Detection (CLASP-Ar)
Bhuvanesh Verma, Ali Abusaleh and Alexander Mehler
Arabic-
language stance detection remains challenging, and previous shared-task systems have largely relied on multitask learning and ensembles. While these systems achieve state-of-the-art performance, their applicability and transferability are limited by the additional complexity introduced by multitask learning.To reduce this complexity, we introduce $\texttt{CLASP-Ar}$, which reformulates the task as cloze-style masked
language modeling. In this approach, the target, predicted sentiment, and text are combined into a single
prompt whose $\texttt{[MASK]}$ prediction is restricted to a verbalizer-constrained label vocabulary.
Read the full abstract ▾
Computation and Language
TTLab at AlexandriaX-2026: A Fine-Tuned Surface Tagger for Arabic Machine-Translation Error-Span Detection and Classification
Ali Abusaleh, Bhuvanesh Verma and Alexander Mehler
We present TTLab's submission to the AlexandriaX-2026 Subtask~3 on Arabic MT error span detection and classification. Our system frames the task as
token-level classification over surface forms, preserving character offsets to ensure exact
alignment with the evaluation metric. To handle severe label imbalance, we employ a focal loss with class weighting and dialect-specific decoding thresholds. Among six Arabic pre-trained encoders, MARBERTv2 achieves the best overall performance of 40.8 and 40.91 on the development and test set, respectively, ranking $\nth{3}$ out of all participating teams. While our system localizes error spans effectively, classification of rare error types remains challenging, highlighting the need for data augmentation for tail categories. The code is available at ${\href{https://github.com/ENTAILab/arabic-dialectal-mt-error-span-detection}{\faGithub~ TTLab at AlexandriaX-2026}$
Read the full abstract ▾
Computation and Language · Cross-listed from cs.SE
CodeGraph: Open-Taxonomy Knowledge Graph for Source Code with Wikidata Grounding
Federico Pennino, Andrea Gurioli, Stefano Zacchiroli et al. (+2 more)
Public software repositories, like GitHub and Software Heritage Archive, store billions of files, yet extracting their implicit engineering knowledge —i.e., the
algorithms they implement, the paradigms they follow, the patterns they instantiate, and the application domains they serve— remains challenging, as current tools are constrained to syntactic and
token-level analysis. We present a pipeline for building an open-taxonomy semantic annotation of source code using a code-specialised Large
Language Model. The extracted entities are grounded in Wikidata through a three-stage linking procedure: a deterministic SPARQL stage handles unambiguous entities, a Deep Research
Agent resolves the residual long tail, and a hierarchy-rollup stage imports the parent-of closure of each resolved Wikidata identifier. The resulting annotations are materialised as a source-code-specific open-taxonomy knowledge graph. We further introduce a calibrated quality-assurance protocol that quantifies annotation precision by combining a small human gold set with an LLM-as-a-judge filter. We applied our pipeline to the 167 million files of the Stack-Edu corpus, creating the first known large-scale open-taxonomy knowledge graph for source code. Our graph, named CodeGraph, contains approximately 158 million nodes, which include around 145 million files, about 63,000 extracted concept entities (such as
algorithms, paradigms, design patterns, and application domains), and roughly 19,800 grounded Wikidata entities. Furthermore, CodeGraph features approximately 1 billion typed edges that connect files to their respective concepts, link these concepts to their grounded Wikidata identifiers, and relate them to their parent categories, covering 14 programming
languages.
Read the full abstract ▾
All authors Federico Pennino, Andrea Gurioli, Stefano Zacchiroli, Maurizio Gabbrielli, Paolo Ferragina
Computation and Language
YODAS v3: Over 1 Million Hours of High-Bandwidth, Stereophonic, Multilingual Speech
William Chen, Shinnosuke Takamichi, Sayaka Shiota et al. (+3 more)
We present YODAS v3, a weakly-labeled
speech corpus containing over 1.1 million hours of 48kHz multi-channel audio in 147
languages, released under a CC BY 3.0 license. YODAS v3 is not only the largest open
speech dataset to date, but also the first truly large-scale
speech corpus with high-fidelity stereo audio. We first provide the collection methodology for the corpus, where we introduce new techniques for gathering
language-balanced
speech data. The effectiveness of our approach is shown by the
language distribution of the crawled data: 22
languages in YODAS v3 have over 10K hours and 73
languages have over 5K hours of data. We then conduct extensive analyses on the composition of the data, such as the distribution of
languages, audio quality, and transcription quality. Finally, we train baseline
speech recognition and neural codec models to show the effectiveness of the
dataset. Download at https://huggingface.co/
datasets/espnet/yodas3.
Read the full abstract ▾
All authors William Chen, Shinnosuke Takamichi, Sayaka Shiota, Satoru Fukayama, Samuele Cornell, Shinji Watanabe
Computation and Language
Two Emojis of Difference: What Multilingual Affective Generation Benchmarks Actually Measure
Fardeen Sadab and Adib Sakhawat
We audit a multilingual affective generation
benchmark eight instruction-tuned LLMs producing emoji summaries for 17,100 Bangla, English and Hindi sentences, with 6,960 human judgements and find its headline conclusions to be artefacts of the measurement instrument rather than properties of the systems. Treating annotators as a random rather than a fixed factor, no system differs significantly from any other ($F(7,14)=0.59$, $p=0.76$), although the conventional analysis declares 19 of 28 pairwise differences significant. Annotator identity explains far more rating variance than system identity, and the winning system changes whenever any single annotator is removed. The ordering that does emerge tracks output length: mean emoji count explains 78.7% of between-system variance, and a within-item length-matched comparison over 2,599 pairs reverses the leaderboard. We further show that cross-provider anisotropy differences vanish under mean-centring, that per-
language token costs change sign with the normalising unit, and that multi-view row-wise splits inflate macro-F1 by $3.1$ points and change the top-ranked system. In place of preference scoring we propose **emoji-affect decodability**, a reference-based probe whose rankings are stable to $\pm0.003$ macro-F1 across seeds.
Read the full abstract ▾
Computation and Language
IterSynth: Rethinking Deep Search Agents via Role-Decoupled Iterative Synthesis
Xingyu Wu, Yuchen Yan, Zhengxi Lu et al. (+9 more)
Deep search requires LLM
agents to decompose complex queries, search for evidence, and synthesize grounded answers, yet existing ReAct-style
agents suffer from two limitations: role coupling, where one policy must handle planning, evidence use, and synthesis; and context accumulation, where growing search histories introduce noise and obscure useful information. To address these issues, we propose IterSynth, a role-decoupled and summary-based paradigm that alternates between a Planner for identifying information needs and a Synthesizer for integrating evidence into an evolving summary state. This design separates planning from synthesis while using the summary as the persistent state of search, reducing both capability coupling and context noise. To train IterSynth effectively, we further introduce Role-Decoupled Policy Optimization (RDPO) for reinforcement learning, which combines terminal outcome rewards with turn-level rubric evaluations and computes role-specific advantages for more precise credit assignment. Experiments on five long-horizon deep-search
benchmarks such as BrowseComp and Xbench-DS show that IterSynth-8B achieves an average score of 50.7, surpassing the strongest prior $\leq$8B
agent by +4.2%. Moreover, IterSynth serves as a model-agnostic prompting paradigm, delivering substantial zero-shot gains over ReAct and similar prompting paradigms on frontier proprietary models.
Read the full abstract ▾
All authors Xingyu Wu, Yuchen Yan, Zhengxi Lu, Siqi Chen, Xin ZHANG, Aiting Liu, Chao Deng, Jie Liu, Jin Ma, Jian Shao, Jun Xiao, Yongliang Shen
Computation and Language · Cross-listed from cs.AI
Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures
Ruoqi Guo, Yi Liu, Gelei Deng et al. (+6 more)
Detectors of
alignment failures screen deployed
language models and score
alignment benchmarks. Most are generative judges that spend a decoding pass on every criterion, and classifiers that read
token probabilities, such as Llama Guard, still score one fixed label per call. Jev, a model trained with reinforcement learning for calibrated decisions (RLCD), answers many typed questions about one input with calibrated probabilities in a single call. Whether it detects
alignment failures has not been measured. We present RLCDAlignBench, which
benchmarks Jev on ten
alignment failures: sycophancy, jailbreaks, deception,
prompt injection,
hallucination,
privacy violation, social bias, reward hacking, concealing uncertainty, and power seeking. It spans 44
benchmarks and five target models, labelled by each
benchmark's scorer and, on two, by humans. Many of these failures are relational, defined against a reference, such as the user's belief or an injected instruction, that the response alone does not reveal. Our key idea is therefore to vary what Jev is asked separately from what it sees: the question's wording and answer type on one side, the fields of the input on the other. A single generic question reaches a median AUROC of 0.886 zero-shot and beats supervised baselines on most
benchmarks. Question wording matters little, while context matters more, mostly through fields that encode the label. Jev matches the reference scorer's agreement with human labels, surfaces label defects in existing
benchmarks, and costs 63x less than LLM-judge scorers. Code and data: https://github.com/sumleo/RLCDAlignBench.
Read the full abstract ▾
All authors Ruoqi Guo, Yi Liu, Gelei Deng, Yuekang Li, Lida Zhao, Yutao Wu, Simin Chen, Ying Zhang, Leo Yu Zhang
Computation and Language
agentic-ger: terminology recovery in long-form speech using global context
Yanqiao Zhu, Wupeng Wang, Zhifu Gao et al. (+2 more)
Recent advances in
speech language models have improved automatic
speech recognition (ASR) for long-form audio. However, accurately and consistently transcribing domain-specific terminology remains challenging. Motivated by the world knowledge and contextual capability of large
language models (LLMs), we propose Agentic-GER, an LLM-based
agent for terminology correction in long-form
speech. The
agent uses global context from the full transcript to identify suspicious terms and resolve ambiguous hypotheses. It selectively re-transcribes the source
speech to check candidate corrections, and uses accepted edits to guide subsequent decisions. Experiments with four LLMs and two ASR systems on GigaSpeechBench show consistent terminology improvements in both Chinese and English, with and without thinking. On Chinese
speech, Agentic-GER achieves up to a 36.8% relative reduction in biased character error rate (B-CER) over the Whisper baseline.
Read the full abstract ▾
All authors Yanqiao Zhu, Wupeng Wang, Zhifu Gao, Xiangang Li, Xie Chen
Computation and Language
Rufus-Air: An Open LLM Post-Training Recipe
Chia-Yuan Chang, Renyuan Cheng, Rui Feng et al. (+19 more)
Rufus-Air is an open and reproducible post-training recipe on GLM-4.5-Air-Base (106B-A12B), organized as a serial pipeline of eight stages: SFT,
Reasoning RL, Coding RL, Instruction-Following RL, General
Agent, Coding
Agent, Search
Agent, and RLHF. We document the data, reward design, infrastructure, stage order, and stagewise results needed to reproduce the recipe. Stages progress from basic to advanced capabilities and from hard, verifiable rewards to softer judge-based signals. Training builds on open-source components and public data, much of it used as released, without new human annotation or an in-house
distillation teacher. Our main findings are that (i) diverse, high-quality SFT establishes a strong capability floor; (ii) difficulty filtering keeps RL
prompts within a productive learning range; (iii) reward reliability provides a practical principle for ordering stages; and (iv) infrastructure and engineering choices are part of the recipe, not just an implementation detail. Rufus-Air improves over the official GLM-4.5-Air post-trained release and is competitive with similarly sized open models.
Read the full abstract ▾
All authors Chia-Yuan Chang, Renyuan Cheng, Rui Feng, Xiaotian Han, Yuan He, Hongye Jin, Linwei Li, Shiyang Li, Fenglin Liu, Xin Liu, Priyanka Nigam, Haoyang Wen, Zhenghao Xu, Zhuocheng Xu, Bing Yin, Qingyu Yin, Chao Zhang, Rongzhi Zhang, Zhihan Zhang, Zixuan Zhang, Zixuan Zhang, Tuo Zhao
Computation and Language
Controlling Backchannels in Streamable Full-duplex Models
Maike Züfle, Peter Polák, Sefik Emre Eskimez et al. (+3 more)
Backchannels, brief acknowledgements like "uh-huh" produced while the other party may still be talking, are central to natural conversation, but full-duplex spoken dialogue models rarely model them explicitly. We introduce a lightweight backchannel head that predicts, from a full-duplex model's own hidden states, when a backchannel should begin. Once this probability crosses a tunable threshold, a backchannel is force-decoded. Attached to both a 7B (PersonaPlex) and a 1B (F-Actor) model, it generalizes across scale. Probing confirms the hidden states anticipate real human timing, and generation evaluation shows more frequent, better-timed backchannels. Human raters judge the resulting backchannels on par with real ones.
Read the full abstract ▾
All authors Maike Züfle, Peter Polák, Sefik Emre Eskimez, Jan Niehues, Peter Bell, Ondřej Klejch
Computation and Language
Large Language Models for Programming: Actually Fixing or Reimplementing Incorrect Code?
Alexandru Stefan Stoica, Traian Rebedea and Marian Cristian Mihaescu
Recent studies have shown that Large
Language Models can effectively solve problems and fix bugs in diverse programming environments, including competitive programming. Existing approaches primarily evaluate LLM performance in problem solving or bug fixing independently, but do not explore the relationship between these two capabilities. This work focuses on determining how much the LLM deviates from a buggy solution to fix the bug compared to a human-written patch, and if there is a bias towards generating entirely new solutions. We construct a
dataset with all the submissions ($\sim$ 3000) from a couple of users from Codeforces, and we match each buggy submission with its corresponding human fix. By using the similarity between the buggy solution and the human fix as a baseline, we evaluate the quality of LLM-generated bug fixes on 3 OpenAI GPT models (gpt-5-nano, gpt-5-mini, gpt-5.1). We check if the generated solutions solve the problem by using the Codeforces-R1
dataset, an openly available
dataset that has tests generated with the DeepSeek-R1 model. Our findings suggest that LLMs tend to modify more lines than necessary compared to human fixes and, in some cases, generate entirely new solutions. We also observe that LLMs solve more problems correctly when allowed to generate solutions from scratch rather than patch buggy submissions, even when those submissions are close to the human patch. This has important implications for the design of AI-assisted programming tools, particularly in supporting user debugging processes and promoting incremental problem-solving strategies rather than solution replacement.
Read the full abstract ▾
Computation and Language
Baseline Shape Decides the Verdict: A Controlled Re-Examination of Ternary Language Models at 60K Parameters
Gautam Veldanda
Ternary (1.58-bit) weights are attractive for microcontroller-class
language models, but the sub-1M-parameter regime rests mainly on isolated, single-seed comparisons. One prominent example reports that a routed ternary block (convolution, diagonal SSM and sparse
attention mixed by a per-
token router) beats a parameter-matched full-precision
transformer by 22% at 60K parameters, attributing this to inductive bias. We re-run it under one fixed recipe, three seeds per
cell, 98 byte-level runs on one laptop. (i) Baseline shape dominates: at a 16M-byte budget, param-matched
transformers span 22.6% in validation loss purely by depth/width choice - far more than any architecture effect we measure there - and the best-shaped
transformer ties the routed model, so the published margin is at least partly a baseline-shape effect; the ordering of shapes reverses with budget, so no single fixed shape can be trusted. (ii) At 130M bytes the routed model does win, by 22.2-24.0% over the three
transformer shapes we evaluate there - but a plain gated diagonal-SSM block beats it by a further 9.1%, and the routed model's own router puts most of its weight on its recurrent pathway, so the gain does not require routing. (iii) The ternary penalty differs by architecture at the larger budget (+5.3% best
transformer vs. +19.5% routed, +28.1% gated SSM), but we cannot attribute that to architecture alone: our
transformers keep learned positional
embeddings in full precision, 11-22% of their parameters, so they are less quantized than the models they are compared with. (iv) A 90/10 full-precision-then-ternary schedule beats all-ternary training, but only at a stage-2 learning rate about 10x the pretraining peak; at a conventional fine-tuning rate it looks 15.3% worse, reversing the conclusion. The from-scratch baseline was not itself learning-rate tuned, which bounds (iii) and (iv). Code and run logs released.
Read the full abstract ▾
Computation and Language
Likelihood Ranking doesn't Scale Like Prompting in LLMs
Alessandro Bondielli, Lucia Passaro, Davide Bacciu et al. (+1 more)
LLM evaluation is commonly performed either by prompting models to produce answers or by scoring candidate outputs with likelihood-based metrics. In multiple-choice QA, however, standard likelihood-based scoring is still conditioned on the question and answer set, and can therefore leverage the same task-conditioned answer-selection interface used in prompting. We study a complementary protocol based on likelihood ranking of declarative statements constructed from the same question–answer pairs. Across 95 decoder-only models, ranging from 0.1B to 104B parameters, and 10 MCQA
datasets, we find a systematic divergence between declarative-statement likelihood ranking and prompted answering. Statement-likelihood accuracy remains comparatively stable across scale, whereas prompted answering improves sharply with scale and instruction-tuning. These results suggest that likelihood preferences over controlled declarative alternatives and task-conditioned answer selection probe distinct aspects of model behavior, and should not be treated as interchangeable.
Read the full abstract ▾
All authors Alessandro Bondielli, Lucia Passaro, Davide Bacciu, Alessandro Lenci
Computation and Language
BanglaTurn: A Benchmark and Whisper-Based Model for End-of-Turn Detection in Bangla Speech
Mizbaul Haque Maruf
This paper presents BanglaTurn, a corpus for end-of-turn detection in Bangla conversational
speech, and a model trained on it. The corpus holds 35,374 samples of 3 to 15 s of podcast
speech, labelled for turn state by combining speaker diarization with an LLM pass, with every label then checked by a human annotator. The model pairs a Whisper encoder with task-specific classification heads. On a class-balanced test set drawn from a held-out podcast, it reaches 84.33% accuracy (95% CI 80.3 to 88.1) against 69.28% for the Smart-Turn v3 baseline, and lowers the false negative rate from 51.57% to 7.55% at the cost of a higher false positive rate. We report what encoder layer fine-tuning, multi-scale pooling and INT8 quantization each contribute, and latency stays within 165 to 191 ms end to end on CPU.
Read the full abstract ▾
Computation and Language
From Policy Documents to Structured Survey Responses: Evaluating Large Language Models for Policy Monitoring
Carolyn Cole, Matthias Deschryvere, Toqeer Ehsan et al. (+1 more)
Science, technology, and innovation policies are crucial for competitiveness, yet their diversity and scale make them difficult to map and monitor consistently. Existing approaches rely heavily on manual survey efforts, which are costly and challenging to scale across countries. Large
language models (LLMs) enable new possibilities for extracting and structuring information from long and unstructured policy documents. This paper presents an application of LLMs as "AI respondents" for generating structured survey responses from policy texts. We develop a data extraction pipeline based on long-context in-context learning to map information from public web sources into predefined survey categories, including policy instruments, target groups, and thematic areas. The pipeline integrates a validation step using a secondary LLM to assess relevance and evidence, alongside comparisons with human-provided responses. Using a multi-country
dataset, we evaluate the
alignment between LLM-generated and human-generated outputs through overlap measures and cross-validation. Results show that LLMs achieve high agreement for structured indicators (84-95%), while differences remain in free-text fields, where models tend to provide more detailed procedural descriptions. These findings highlight the potential of hybrid human-AI workflows for policy monitoring, improving both efficiency and scalability while maintaining the need for human validation and contextual interpretation.
Read the full abstract ▾
All authors Carolyn Cole, Matthias Deschryvere, Toqeer Ehsan, Arash Hajikhani
Computation and Language
Parts-of-Speech as Emergent Categories in SAE Latent Space
Alessandro Bondielli, Lucia Passaro, Serena Auriemma et al. (+1 more)
Sparse AutoEncoders (SAEs) offer a promising way to inspect
language model representations, but it is still unclear what kind of linguistic structure their latents expose. We use part-of-
speech (PoS) categories as a controlled test case to study whether morpho-syntactic information is encoded by individual latents or by structured groups of features. We find that PoS distinctions are highly recoverable from SAE activations, but do not align with one-to-one latent / category mappings. This recoverability is not reducible to lexical memorisation, and Open and Closed PoS classes differ substantially. Categories are supported by compact groups of sparse latents, with substantial variation across tags. These groups remain stable on held-out data, while also showing overlap between related categories. Our results show that SAEs localise morpho-syntactic information in a distributed and category-dependent form rather than through atomic grammatical features.
Read the full abstract ▾
All authors Alessandro Bondielli, Lucia Passaro, Serena Auriemma, Alessandro Lenci
Computation and Language
ArGuard Shared Task: Harmful Content Detection in Arabic Memes and LLM Prompts
Firoj Alam, Md. Rafiul Biswas, Mohamed Bayan Kmainasi et al. (+5 more)
ArGuard is a shared task on harmful content detection in Arabic memes and LLM
prompts. It includes two tracks: Track A focuses on
multimodal hate detection in Arabic memes, while Track B addresses harmful
prompt detection for Arabic LLM safety evaluation. In total, 58 teams registered, 35 participated in the final evaluation, and 27 submitted system-description papers. Participating teams explored models such as AraBERT, Jais, and Qwen3-VL. The best systems achieved macro-F1 scores of 0.823 on A1, 0.419 on A2, 0.984 on B1, and 0.790 on B2. Fine-grained meme classification in A2 was the most challenging setting, partly due to sparse labels and train-test distribution shifts.
Read the full abstract ▾
All authors Firoj Alam, Md. Rafiul Biswas, Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Hamdy Mubarak, George Mikros, Abul Hasnat, Wajdi Zaghouani