Computation and Language
Syntax as a Rosetta Stone: Universal Dependencies for In-Context Coptic Translation
Abhishek Purushothama, Emma Thronson, Alexia Guo et al. (+1 more)
Low-resource machine translation requires methods that differ from those used for high-resource
languages. This paper proposes a novel in-context learning approach to support low-resource machine translation of the Coptic
language to English, with syntactic augmentation from Universal Dependencies parses of input sentences. Building on existing work using bilingual dictionaries to support inference for vocabulary items, we add several representations of syntactic analyses to our inputs , specifically exploring the inclusion of raw parser outputs, verbalizations of parses in plain English, and targeted instructions of difficult constructions identified in sub-trees and how they can be translated. Our results show that while syntactic information alone is not as useful as dictionary-based glosses, combining retrieved dictionary items with syntactic information achieves significant gains across model sizes, achieving new state-of-the-art translation results for Coptic.
Read the full abstract ▾
All authors Abhishek Purushothama, Emma Thronson, Alexia Guo, Amir Zeldes
Computation and Language · Cross-listed from cs.LG
Towards Understanding the Robustness of Sparse Autoencoders
Ahson Saiyed, Sabrina Sadiekh and Chirag Agarwal
Large
Language Models (LLMs) remain vulnerable to optimization-based jailbreak attacks that exploit internal gradient structure. While Sparse Autoencoders (SAEs) are widely used for interpretability, their robustness implications remain underexplored. We present a study of integrating pretrained SAEs into
transformer residual streams at inference time, without modifying model weights or blocking gradients. Across four model families (Gemma, LLaMA, Mistral, Qwen) and two strong white-box attacks (GCG, BEAST) plus three black-box
benchmarks, SAE-augmented models achieve up to a 5x reduction in jailbreak success rate relative to the undefended baseline and reduce cross-model attack transferability. Parametric ablations reveal (i) a monotonic dose-response relationship between L0 sparsity and attack success rate, and (ii) a layer-dependent defense-utility tradeoff, where intermediate layers balance robustness and clean performance. These findings are consistent with a representational bottleneck hypothesis: sparse projection reshapes the optimization geometry exploited by jailbreak attacks.
Read the full abstract ▾
Computation and Language
Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Models
Lin Yao
Diffusion language models (dLLMs) generate text through iterative denoising, filling multiple masked positions at each step. Positions filled in the same step are predicted without conditioning on one another's newly filled values and can therefore be mutually inconsistent; once retained, these inconsistencies become context for later predictions. We introduce
Token-to-Mask (T2M), a training-free inference-time correction method that identifies low-confidence positions using the model's probability of the current
token, remasks them, and reconstructs them in later denoising steps. On dLLMs equipped with correction mechanisms, a single T2M configuration transfers across tasks and models without retuning and broadly improves task metrics over each model's native correction mechanism. In controlled experiments, we decompose correction methods into a detector that identifies suspicious
tokens and an action that determines how to revise them. Holding the detector fixed, remasking yields higher task metrics than replacement; across the tested detector–action combinations, current-
token-probability detection paired with remasking performs best. Compared with direct editing, T2M converts additional inference compute into performance gains more effectively and, on most tasks, retains a sequential-step advantage over autoregressive
token-by-
token decoding.
Read the full abstract ▾
Computation and Language
Investigating Counterfactual Unfairness in LLMs towards Identities through Humor
Shubin Kim, Yejin Son, Junyeong Park et al. (+6 more)
Humor holds up a mirror to social perception: what we find funny often reflects who we are and how we judge others. When
language models engage with humor, their reactions expose the social assumptions they have internalized from training data. In this paper, we investigate counterfactual unfairness through humor by observing how the model's responses change when we swap who speaks and who is addressed while holding other factors constant. Our framework spans three tasks: humor generation refusal, speaker intention inference, and relational/societal impact prediction, covering both identity-agnostic humor and identity-specific disparagement humor. We introduce interpretable bias metrics that capture asymmetric patterns under identity swaps. Experiments across state-of-the-art models reveal consistent relational disparities: jokes told by privileged speakers are refused up to 67.5% more often, judged as malicious 64.7% more frequently, and rated up to 1.5 points higher in social harm on a 5-point scale. These patterns highlight how sensitivity and stereotyping coexist in generative models, complicating efforts toward
fairness and cultural
alignment.
Read the full abstract ▾
All authors Shubin Kim, Yejin Son, Junyeong Park, Keummin Ka, Seungbeen Lee, Jaeyoung Lee, Hyeju Jang, Alice Oh, Youngjae Yu
Computation and Language
Scripts Through Time: A Survey of the Evolving Role of Transliteration in NLP
Thanmay Jayakumar, Deepon Halder and Raj Dabre
Cross-lingual transfer in NLP is often hindered by the “script barrier” where differences in writing systems inhibit transfer learning between
languages. Transliteration, the process of converting the script, has emerged as a powerful technique to bridge this gap by increasing lexical overlap. This paper provides a comprehensive survey of the application of transliteration in cross-lingual NLP. We present a taxonomy of key motivations to utilize transliterations in
language models, and provide an overview of different approaches of incorporating transliterations as input. We analyze the
evolution and effectiveness of these methods, discussing the critical trade-offs involved, and contextualize their need in modern LLMs. The review explores various settings that show how transliteration is beneficial, including handling code-mixed text, leveraging
language family relatedness, and pragmatic gains in inference efficiency. Based on this analysis, we provide concrete recommendations for researchers on selecting and implementing the most appropriate transliteration strategy based on their specific
language, task, and resource constraints.
Read the full abstract ▾
Computation and Language
Characterizing AlphaEarth Embedding Geometry for Agentic Environmental Reasoning
Mashrekur Rahman, Samuel J. Barrett and Christina Last
Earth observation foundation models encode land surface information into dense
embedding vectors, yet the
geometric structure of these representations and its implications for downstream
reasoning remain underexplored. We characterize the
manifold geometry of Google AlphaEarth's 64-dimensional
embeddings across 12.1 million Continental United States samples (2017–2023) and develop an agentic system that leverages this
geometric understanding for environmental
reasoning. The
manifold is non-Euclidean: effective dimensionality is 13.3 (participation ratio) from 64 raw dimensions, with local intrinsic dimensionality of approximately 10. Tangent spaces rotate substantially, with 84% of locations exceeding 60\textdegree{} and local-global
alignment (mean$|\cos\theta| = 0.17$) approaching the random baseline of 0.125. Supervised linear probes indicate that concept directions rotate across the
manifold, and compositional vector arithmetic using both PCA-derived and probe-derived directions yields poor precision.
Retrieval instead produces physically coherent results, with local geometry predicting
retrieval coherence ($R^2 = 0.32$). Building on this characterization, we introduce an agentic system with nine specialized tools that decomposes environmental queries into
reasoning chains over a FAISS-indexed
embedding database. A five-condition ablation (120 queries, three complexity tiers) shows that
embedding retrieval dominates response quality ($\mu = 3.79 \pm 0.90$ vs.\ $3.03 \pm 0.77$ parametric-only; scale 1–5), with peak performance on multi-step comparisons ($\mu = 4.28 \pm 0.43$). A cross-model
benchmark show that
geometric tools reduce Sonnet 4.5's score by 0.12 points but improve Opus 4.6's by 0.07, with Opus achieving higher
geometric grounding (3.38 vs.\ 2.64), suggesting that the value of
geometric characterization scales with the
reasoning capability of the consuming model.
Read the full abstract ▾
Computation and Language
Probing for Reading Times
Eleftheria Tsipidi, Samuel Kiegeland, Francesco Ignazio Re et al. (+4 more)
Probing has shown that
language model representations encode rich linguistic information, but it remains unclear whether they also capture cognitive signals about human processing. In this work, we probe
language model representations for human reading times. Using regularized linear regression on two eye-tracking corpora spanning five
languages (English, Greek, Hebrew, Russian, and Turkish), we compare the representations from every model layer against scalar predictors – surprisal, information value, and logit-lens surprisal. We find that the representations from early layers outperform surprisal in predicting early-pass measures such as first fixation and gaze duration. The concentration of predictive power in the early layers suggests that human-like processing signatures are captured by low-level structural or lexical representations, pointing to a functional
alignment between model depth and the temporal stages of human reading. In contrast, for late-pass measures such as total reading time, scalar surprisal remains superior, despite its being a much more compressed representation. We also observe performance gains when using both surprisal and early-layer representations. Overall, we find that the best-performing predictor varies strongly depending on the
language and eye-tracking measure.
Read the full abstract ▾
All authors Eleftheria Tsipidi, Samuel Kiegeland, Francesco Ignazio Re, Tianyang Xu, Mario Giulianelli, Karolina Stanczak, Ryan Cotterell
Computation and Language · Cross-listed from cs.CR
Beyond Indistinguishability: Measuring Extraction Risk in LLM APIs
Ruixuan Liu, David Evans and Li Xiong
Indistinguishability properties such as differential
privacy bounds or low empirically measured membership inference are widely treated as proxies to show a model is sufficiently protected against broader memorization risks. However, we show that indistinguishability properties are neither sufficient nor necessary for preventing data extraction in LLM APIs. We formalize a
privacy-game separation between extraction and indistinguishability-based
privacy, showing that indistinguishability and inextractability are incomparable: upper-bounding distinguishability does not upper-bound extractability. To address this gap, we introduce $(l, b)$-inextractability as a definition that requires at least $2^b$ expected queries for any black-box adversary to induce the LLM API to emit a protected $l$-gram substring. We instantiate this via a worst-case extraction game and derive a rank-based extraction risk upper bound for targeted exact extraction, as well as extensions to cover untargeted and approximate extraction. The resulting
estimator captures the extraction risk over multiple attack trials and prefix adaptations. We show that it can provide a tight and efficient estimation for standard greedy extraction and an upper bound on the probabilistic extraction risk given any decoding configuration. We empirically evaluate extractability across different models, clarifying its connection to distinguishability, demonstrating its advantage over existing extraction risk
estimators, and providing actionable mitigation guidelines across model training, API access, and decoding configurations in LLM API deployment. Our code is publicly available at: https://github.com/Emory-AIMS/Inextractability.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CR
Owner-Harm: A Missing Threat Model for AI Agent Safety
Dongcheng Zhang and Yiqing Jiang
Existing AI
agent safety
benchmarks focus on generic criminal harm (cybercrime, harassment, weapon synthesis), leaving a systematic blind spot for a distinct and commercially consequential threat category:
agents harming their own deployers. Real-world incidents illustrate the gap: Slack AI credential exfiltration (Aug 2024), Microsoft 365 Copilot calendar-injection leaks (Jan 2024), and a Meta
agent unauthorized forum post exposing operational data (Mar 2026). We propose Owner-Harm, a formal threat model with eight categories of
agent behavior damaging the deployer. We quantify the defense gap on two
benchmarks: a compositional safety system achieves 100% TPR / 0% FPR on AgentHarm (generic criminal harm) yet only 14.8% (4/27; 95% CI: 5.9%-32.5%) on AgentDojo injection tasks (
prompt-injection-mediated owner harm). A controlled generic-LLM baseline shows the gap is not inherent to owner-harm (62.7% vs. 59.3%, delta 3.4 pp) but arises from environment-bound symbolic rules that fail to generalize across tool vocabularies. On a post-hoc 300-scenario owner-harm
benchmark, the gate alone achieves 75.3% TPR / 3.3% FPR; adding a deterministic post-audit verifier raises overall TPR to 85.3% (+10.0 pp) and Hijacking detection from 43.3% to 93.3%, demonstrating strong layer complementarity. We introduce the Symbolic-Semantic Defense Generalization (SSDG) framework relating information coverage to detection rate. Two SSDG experiments partially validate it: context deprivation amplifies the detection gap 3.4x (R = 3.60 vs. R = 1.06); context injection reveals structured goal-action
alignment, not text concatenation, is required for effective owner-harm detection.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.DC
Unlocking the Edge deployment and ondevice acceleration of multi-LoRA enabled one-for-all foundational LLM
Sravanth Kodavanti, Sowmya Vajrala, Srinivas Miriyala et al. (+13 more)
Deploying large
language models (LLMs) on smartphones poses significant engineering challenges due to stringent constraints on
memory, latency, and runtime flexibility. In this work, we present a hardware-aware framework for efficient on-device inference of a LLaMA-based
multilingual foundation model supporting multiple use cases on Samsung
Galaxy S24 and S25 devices with SM8650 and SM8750 Qualcomm chipsets respectively. Our approach integrates application-specific LoRAs as runtime inputs to a single frozen inference graph, enabling dynamic task switching without recompilation or
memory overhead. We further introduce a multi-stream decoding mechanism that concurrently generates stylistic variations - such as formal, polite, or jovial responses - within a single forward pass, reducing latency by up to 6x. To accelerate
token generation, we apply Dynamic Self-Speculative Decoding (DS2D), a tree-based strategy that predicts future
tokens without requiring a draft model, yielding up to 2.3x speedup in decode time. Combined with quantization to INT4 and architecture-level optimizations, our system achieves 4-6x overall improvements in
memory and latency while maintaining accuracy across 9
languages and 8 tasks. These results demonstrate practical feasibility of deploying multi-use-case LLMs on edge devices, advancing the commercial viability of Generative AI in mobile platforms.
Read the full abstract ▾
All authors Sravanth Kodavanti, Sowmya Vajrala, Srinivas Miriyala, Utsav Tiwari, Uttam Kumar, Utkarsh Kumar Mahawar, Achal Pratap Singh, Arya D, Narendra Mutyala, Vikram Nelvoy Rajendiran, Sharan Kumar Allur, Euntaik Lee, Dohyoung Kim, HyeonSu Lee, Gyusung Cho, JungBae Kim
Computation and Language · Cross-listed from cs.LG
Sessa: Selective State Space Attention
Liubomyr Horbatko
Modern sequence modeling is dominated by two families:
Transformers, whose self-
attention can access arbitrary elements of the visible sequence, and structured state-space models, which propagate information through an explicit recurrent state. These mechanisms face different limitations on long contexts: when
attention is diffuse, the influence of individual
tokens is diluted across the effective support, while recurrent state propagation can lose long-range sensitivity unless information is actively preserved. As a result, both mechanisms face challenges in preserving and selectively retrieving information over long contexts. We propose Sessa, a decoder that places
attention inside a recurrent feedback path. This creates many
attention-based paths through which past
tokens can influence future states, rather than relying on a single
attention read or a single recurrent chain. We prove that, under explicit assumptions and matched regimes, Sessa admits power-law
memory tails $O(\ell^{-\beta})$ for $0 < \beta < 1$, with slower decay than in the corresponding
Transformer and Mamba-style baselines. We further give an explicit construction that achieves this power-law rate. Under the same assumptions, Sessa is the only model class among those considered that realizes flexible selective
retrieval, including profiles whose influence does not decay with distance. Consistent with this theoretical advantage, across matched experiments, Sessa achieves the strongest performance on long-context
benchmarks while remaining competitive with
Transformer and Mamba-style baselines on short-context
language modeling.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
A multimodal and temporal foundation model for virtual patient representations at healthcare system scale
Andrew Zhang, Tong Ding, Sophia J. Wagner et al. (+8 more)
Modern medicine generates vast
multimodal data across siloed systems, yet no existing model integrates the full breadth and temporal depth of the clinical record into a unified patient representation. We introduce Apollo, a
multimodal temporal foundation model trained and evaluated on over three decades of longitudinal hospital records from a major US hospital system, composed of 25 billion records from 7.2 million patients, representing 28 distinct medical modalities and 12 major medical specialties. Apollo learns a unified representation space integrating over 100 thousand unique medical events in our clinical vocabulary as well as images and clinical text. This "atlas of medical concepts" forms a computational substrate for modeling entire patient care journeys comprised of sequences of structured and unstructured events, which are compressed by Apollo into virtual patient representations. To assess the potential of these whole-patient representations, we created 322 prognosis and
retrieval tasks from a held-out test set of 1.4 million patients. We demonstrate the generalized clinical
forecasting potential of Apollo
embeddings, including predicting new disease onset risk up to five years in advance (95 tasks), disease progression (78 tasks), treatment response (59 tasks), risk of treatment-related adverse events (17 tasks), and hospital operations endpoints (12 tasks). Using feature attribution techniques, we show that model predictions align with clinically-interpretable
multimodal biomarkers. We evaluate semantic similarity search on 61
retrieval tasks, and moreover demonstrate the potential of Apollo as a
multimodal medical search engine using text and image queries. Together, these modeling capabilities establish the foundation for computable medicine, where the full context of patient care becomes accessible to computational
reasoning.
Read the full abstract ▾
All authors Andrew Zhang, Tong Ding, Sophia J. Wagner, Caiwei Tian, Ming Y. Lu, Rowland Pettit, Joshua E. Lewis, Alexandre Misrahi, Dandan Mo, Long Phi Le, Faisal Mahmood
Computation and Language · Cross-listed from cs.LG
Latent Phase-Shift Rollback: Inference-Time Error Correction via Residual Stream Monitoring and KV-Cache Steering
Manan Gupta and Dhruv Kumar
Large
language models frequently commit unrecoverable
reasoning errors mid-generation: once a wrong step is taken, subsequent
tokens compound the mistake rather than correct it. We introduce $\textbf{Latent Phase-Shift Rollback}$ (LPSR): at each generation step, we monitor the residual stream at a critical layer lcrit, detect abrupt directional reversals (phase shifts) via a cosine-similarity $+$
entropy dual gate, and respond by rolling back the KV-cache and injecting a pre-computed steering vector. No fine-tuning, gradient computation, or additional forward passes are required. LPSR achieves $\mathbf{44.0\%}$ on MATH-500 with an 8B model versus $28.8\%$ for standard AR ($+15.2$ pp; McNemar $\chi^2 = 66.96$, $p < 10^{-15}$). Critically, prompted self-correction, the most natural inference-time baseline, scores only $19.8\%$, below standard AR; LPSR exceeds it by $+24.2$ pp ($\chi^2 = 89.4$, $p \approx 0$). LPSR also outperforms Best-of-16 ($+7.8$ pp) at $5.4\times$ lower
token cost, and surpasses a standard 70B model ($35.2\%$) with $8.75\times$ fewer parameters at ${\sim}3\times$ the
token budget. A 32-layer sweep reveals a novel detection-correction dissociation: error-detection AUC peaks at layer~14 ($0.718$) but task accuracy peaks at layer~16 ($44.0\%$ vs.\ $29.2\%$), demonstrating that optimal monitoring depth differs for detection and correction.
Read the full abstract ▾
Computation and Language
Dual Alignment Between Language Model Layers and Human Sentence Processing
Tatsuki Kuribayashi, Alex Warstadt, Yohei Oseki et al. (+1 more)
A recent study (Kuribayashi et al., 2025) has shown that human sentence processing behavior, typically measured on syntactically unchallenging constructions, can be effectively modeled using surprisal from early layers of large
language models (LLMs). This raises the question of whether such advantages of internal layers extend to more syntactically challenging constructions, where surprisal has been reported to underestimate human cognitive effort. In this paper, we begin by exploring internal layers that better estimate human cognitive effort observed in syntactic ambiguity processing in English. Our experiments show that, in contrast to naturalistic reading, later layers better estimate such a cognitive effort, but still underestimate the human data. This dual
alignment sheds light on different modes of sentence processing in humans and LMs: naturalistic reading employs a somewhat weak prediction akin to earlier layers of LMs, while syntactically challenging processing requires more fully-contextualized representations, better modeled by later layers of LMs. Motivated by these findings, we also explore several probability-update measures using shallow and deep layers of LMs, showing a complementary advantage to single-layer's surprisal in reading time modeling.
Read the full abstract ▾
All authors Tatsuki Kuribayashi, Alex Warstadt, Yohei Oseki, Ethan Gotlieb Wilcox
Computation and Language
GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling
Alireza Dadgarnia, Soroush Tabesh, Mahdi Nikdan et al. (+4 more)
Quantization has become a standard tool for efficient LLM deployment, especially for local inference, where models are now routinely served at 2-3 bits per parameter. The state of the art is currently split into simple scalar quantization techniques, such as GPTQ or AWQ, which are widely deployed but plateau in accuracy at 3-4 bits per parameter (bpp), and "second-generation" vector- or trellis-quantized methods, such as QTIP, GPTVQ and AQLM, which push the accuracy frontier but are notoriously hard to implement and to scale. In this paper, we ask whether this gap is fundamental, or whether a carefully optimized $\textit{scalar}$ quantizer can recover most of it. We answer in the affirmative, by introducing GSQ (Gumbel-Softmax Quantization), a post-training scalar quantization method which jointly learns the per-coordinate grid assignments and the per-group scales using a Gumbel-Softmax relaxation of the discrete grid. GSQ matches the cardinality of the relaxation to the small number of levels available in the target bit-width regime (e.g., 3-8 levels for ternary and 3 bpp, respectively), making optimization tractable. Practically, on the standard Llama-3.1-8B/70B-Instruct models, GSQ closes most of the gap between scalar quantization and the QTIP frontier at 2 and 3 bits, while using a symmetric scalar grid with group-wise quantization, and thus remains compatible with existing scalar inference kernels. We further show that the same discrete-assignment optimization can be applied to practical GGUF K-Quant checkpoints: starting from publicly released GGUF models, GSQ improves accuracy while projecting the result back into the same deployment format. Finally, GSQ scales to trillion-scale Mixture-of-Experts models such as Kimi-K2.5, where vector-quantized methods are difficult to apply. The source code is publicly available at https://github.com/IST-DASLab/GSQ.
Read the full abstract ▾
All authors Alireza Dadgarnia, Soroush Tabesh, Mahdi Nikdan, Michael Helcig, Eldar Kurtic, Maximilian Kleinegger, Dan Alistarh
Computation and Language · Cross-listed from stat.ML
FUSE: Ensembling Verifiers with Zero Labeled Data
Joonhyuk Lee, Virginia Ma, Sarah Zhao et al. (+4 more)
Verification of model outputs is rapidly emerging as a key primitive for both training and real-world deployment of large
language models (LLMs). In practice, this often involves using imperfect LLM judges and reward models since ground truth acquisition can be time-consuming and expensive. We introduce Fully Unsupervised Score Ensembling (FUSE), a method for improving verification quality by ensembling verifiers without access to ground truth correctness labels. The key idea behind FUSE is to control conditional dependencies between verifiers in a manner that improves the unsupervised performance of a class of spectral
algorithms from the ensembling literature. Despite requiring zero ground truth labels, FUSE typically matches or improves upon semi-supervised alternatives in test-time scaling experiments with diverse sets of generator models, verifiers, and
benchmarks. In particular, we validate our method on both conventional academic
benchmarks such as GPQA Diamond and on frontier, unsaturated
benchmarks such as Humanity's Last Exam and IMO Shortlist questions.
Read the full abstract ▾
All authors Joonhyuk Lee, Virginia Ma, Sarah Zhao, Yash Nair, Asher Spector, Regev Cohen, Emmanuel J. Candès
Computation and Language · Cross-listed from cs.AI
ClawEnvKit: Automatic Environment Generation for Claw-Like Agents
Xirui Li, Ming Li, Ion Stoica et al. (+2 more)
Constructing environments for training and evaluating claw-like
agents remains a manual, human-intensive process that does not scale. We argue that what is needed is not just a
dataset, but an automated pipeline capable of generating diverse, verified environments on demand. To this end, we introduce ClawEnvKit, an autonomous generation pipeline that instantiates this formalism from natural
language descriptions. The pipeline comprises three modules: (1) a parser that extracts structured generation parameters from natural
language input; (2) a generator that produces the task specification, tool interface, and scoring configuration; and (3) a validator that enforces feasibility, diversity, structural validity, and internal consistency across the generated environments. Using ClawEnvKit, we construct Auto-ClawEval, the first large-scale
benchmark for claw-like
agents, comprising 1,040 environments across 24 categories. Empirically, Auto-ClawEval matches or exceeds human-curated environments on coherence and clarity at 13,800x lower cost. Evaluated across 4 model families and 8
agent harness frameworks, we find that harness engineering boosts performance by up to 15.7 percentage points over a bare ReAct baseline, completion remains the primary axis of variation with no model saturating the
benchmark, and automated generation enables evaluation at a scale previously infeasible. Beyond static benchmarking, ClawEnvKit enables live evaluation: users describe a desired capability in natural
language and obtain a verified environment on demand, turning evaluation into a continuous, user-driven process. The same mechanism serves as an on-demand training environment generator, producing task distributions that adapt to an
agent's current weaknesses rather than being bounded by existing user logs.
Read the full abstract ▾
All authors Xirui Li, Ming Li, Ion Stoica, Cho-Jui Hsieh, Tianyi Zhou
Computation and Language
Transition-Matrix Regularization for Next Dialogue Act Prediction in Counselling Conversations
Eric Rudolph, Philipp Steigerwald and Jens Albrecht
This paper studies how empirical dialogue-flow statistics can be incorporated into Next Dialogue Act Prediction (NDAP). A KL regularization term is proposed that aligns predicted act distributions with corpus-derived transition patterns. Evaluated on a 60-class German counselling taxonomy using 5-fold cross-validation, this improves macro-F1 by 9–42% relative depending on encoder and substantially improves dialogue-flow
alignment. Cross-
dataset validation on HOPE suggests that improvements transfer across
languages and counselling domains. In systematic ablations across pretrained encoders and architectures, the findings indicate that transition regularization provides consistent gains and disproportionately benefits weaker baseline models. The results suggest that lightweight discourse-flow priors complement pretrained encoders, especially in fine-grained, data-sparse dialogue tasks.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CR
Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreaks
Md Rysul Kabir and Zoran Tiganj
Open-weight
language models can be rendered unsafe through several parameter-level interventions, yet models with matched harmful compliance can exhibit fundamentally different failure modes. We compare harmful supervised fine-tuning (SFT), harmful reinforcement learning with verifiable rewards (RLVR), and refusal-feature abliteration in Qwen2.5-7B and Llama-3.1-8B using harmfulness, capability, self-audit, safety reflection, representation similarity, and refusal-direction repair. All three routes reach near-ceiling harmfulness, but SFT causes the broadest capability loss and representational drift; abliteration yields localized, family-dependent refusal-feature suppression; and RLVR largely preserves base-model capability, explicit safety judgments, and representation geometry while retargeting behavior toward compliance. RLVR models consequently remain unusually responsive to safety-reflection
prompts despite complying under direct prompting. We further find that harmful RLVR induces capability-blind compliance: models claim to complete unavailable actions and fabricate information about nonexistent entities. Targeted RLVR calibration substantially reduces this false acceptance without restoring safety or degrading general capability. These results show that harmful compliance, harm recognition, and capability awareness are separable behavioral axes, and that self-audit and
hallucination patterns are not necessarily robust safety signals under adaptive post-training.
Read the full abstract ▾
Computation and Language
MASS-RAG: Multi-Agent Synthesis Retrieval-Augmented Generation
Xingchen Xiao, Heyan Huang, Runheng Liu et al. (+1 more)
Large
language models (LLMs) are widely used in
retrieval-augmented generation (RAG) to incorporate external knowledge at inference time. However, when retrieved contexts are noisy, incomplete, or heterogeneous, a single generation process often struggles to reconcile evidence effectively. We propose MASS-RAG, a multi-
agent synthesis approach to
retrieval-augmented generation that structures evidence processing into multiple role-specialized
agents. MASS-RAG applies distinct
agents for evidence summarization, evidence extraction, and
reasoning over retrieved documents, and combines their outputs through a dedicated synthesis stage to produce the final answer. This design exposes multiple intermediate evidence views, allowing the model to compare and integrate complementary information before answer generation. Experiments on four
benchmarks show that MASS-RAG consistently improves performance over strong RAG baselines, particularly in settings where relevant evidence is distributed across retrieved contexts.
Read the full abstract ▾
All authors Xingchen Xiao, Heyan Huang, Runheng Liu, Jincheng Xie
Computation and Language · Cross-listed from cs.IR
Document-as-Image Representations Fall Short for Scientific Retrieval
Ghazal Khalighinejad, Raghuveer Thirukovalluru, Alexander H. Oh et al. (+1 more)
Many recent document
embedding models are trained on document-as-image representations,
embedding rendered pages as images rather than the underlying source. Meanwhile, existing
benchmarks for scientific document
retrieval, such as ArXivQA and ViDoRe, treat documents as images of pages, implicitly favoring such representations. In this work, we argue that this paradigm is not well-suited for text-rich
multimodal scientific documents, where critical evidence is distributed across structured sources, including text, tables, and figures. To study this setting, we introduce ArXivDoc, a new
benchmark constructed from the underlying LaTeX sources of scientific papers. Unlike PDF or image-based representations, LaTeX provides direct access to structured elements (e.g., sections, tables, figures, equations), enabling controlled query construction grounded in specific evidence types. We systematically compare text-only, image-based, and
multimodal representations across both single-vector and multi-vector
retrieval models. Our results show that: (1) document-as-image representations are consistently suboptimal, especially as document length increases; (2) text-based representations are most effective, even for figure-based queries, by leveraging captions and surrounding context; and (3) interleaved text+image representations outperform document-as-image approaches without requiring specialized training.
Read the full abstract ▾
All authors Ghazal Khalighinejad, Raghuveer Thirukovalluru, Alexander H. Oh, Bhuwan Dhingra
Computation and Language
LQM: Linguistically Motivated Multidimensional Quality Metrics for Machine Translation
Samar M. Magdy, Fakhraddin Alwajih, Abdellah El Mekki et al. (+2 more)
Existing MT evaluation frameworks, including automatic metrics and human evaluation schemes such as Multidimensional Quality Metrics (MQM), are largely
language-agnostic. However, they often fail to capture dialect- and culture-specific errors in diglossic
languages (e.g., Arabic), where translation failures stem from mismatches in
language variety, content coverage, and pragmatic appropriateness rather than surface form alone.We introduce LQM: Linguistically Motivated Multidimensional Quality Metrics for MT. LQM is a hierarchical error taxonomy for diagnosing MT errors through six linguistically grounded levels: sociolinguistics, pragmatics, semantics, morphosyntax, orthography, and graphetics (Figure 1). We construct a bidirectional parallel corpus of 3,850 sentences (550 per variety) spanning seven Arabic dialects (Egyptian, Emirati, Jordanian, Mauritanian, Moroccan, Palestinian, and Yemeni), derived from conversational, culturally rich content. We evaluate six LLMs in a zero-shot setting and conduct expert span-level human annotation using LQM, producing 6,113 labeled error spans across 3,495 unique erroneous sentences, along with severity-weighted quality scores. We complement this analysis with an automatic metric (spBLEU). Though validated here on Arabic, LQM is a
language-agnostic framework designed to be easily applied to or adapted for other
languages. LQM annotated errors data,
prompts, and annotation guidelines are publicly available at https://github.com/UBC-NLP/LQM_MT.
Read the full abstract ▾
All authors Samar M. Magdy, Fakhraddin Alwajih, Abdellah El Mekki, Wesam El-Sayed, Muhammad Abdul-Mageed
Computation and Language · Cross-listed from cs.SD
Aligning Language Models for Lyric-to-Melody Generation with Rule-Based Musical Constraints
Hao Meng, Siyuan Zheng, Shuran Zhou et al. (+2 more)
Large
Language Models (LLMs) show promise in lyric-to-melody generation, but models trained with Supervised Fine-Tuning (SFT) often produce musically implausible melodies with issues like poor rhythm and unsuitable vocal ranges, a phenomenon we term "constraint violation". To address this, we propose a novel
alignment framework that instills musical knowledge without human annotation. We define rule-based musical constraints to automatically generate a preference
dataset from an SFT model's outputs. The model is then aligned through a sequential process, first using Direct Preference Optimization (DPO) on paired preference data, followed by Kahneman-Tversky Optimization (KTO) on unpaired negative samples. Experimental results demonstrate that our aligned model substantially reduces rule violations and outperforms strong baselines in both objective and subjective evaluations, generating melodies with substantially improved musicality and coherence. An interactive demo with audio comparisons is available at https://arain233.github.io/AligningMelody-demo.
Read the full abstract ▾
All authors Hao Meng, Siyuan Zheng, Shuran Zhou, Qiangqiang Wang, Yang Song
Computation and Language
Adversarial Humanities Benchmark: Results on Stylistic Robustness in Frontier Model Safety
Marcello Galisai, Susanna Cifani, Francesco Giarrusso et al. (+5 more)
The
Adversarial Humanities
Benchmark (AHB) evaluates whether model safety refusals survive a shift away from familiar harmful
prompt forms. Starting from harmful tasks drawn from MLCommons AILuminate, the
benchmark rewrites the same objectives through humanities-style transformations while preserving intent. This extends literature on
Adversarial Poetry and
Adversarial Tales from single jailbreak operators to a broader
benchmark family of stylistic obfuscation and goal concealment. In the
benchmark results reported here, the original attacks record 3.84% attack success rate (ASR), while transformed methods range from 36.8% to 65.0%, yielding 55.75% overall ASR across 31 frontier models. Under a European Union AI Act Code-of-Practice-inspired systemic-risk lens, Chemical, biological, radiological and
nuclear (CBRN) is the highest bucket. Taken together, this lack of stylistic robustness suggests that current safety techniques suffer from weak generalization: deep understanding of 'non-maleficence' remains a central unresolved problem in frontier model safety.
Read the full abstract ▾
All authors Marcello Galisai, Susanna Cifani, Francesco Giarrusso, Piercosma Bisconti, Matteo Prandi, Federico Pierucci, Federico Sartore, Daniele Nardi
Computation and Language · Cross-listed from cs.CV
Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation
Jinghui Lu, Jiayi Guan, Zhijian Huang et al. (+47 more)
Chain-of-Thought (CoT)
reasoning has become a powerful driver of trajectory prediction in VLA-based autonomous driving, yet its autoregressive nature imposes a latency cost that is prohibitive for real-time deployment. Latent CoT methods attempt to close this gap by compressing
reasoning into continuous hidden states, but consistently fall short of their explicit counterparts. We suggest that this is due to purely linguistic latent representations compressing a symbolic abstraction of the world, rather than the causal dynamics that actually govern driving. Thus, we present OneVL (One-step latent
reasoning and planning with
Vision-
Language explanations), a unified VLA and World Model framework that routes
reasoning through compact latent
tokens supervised by dual auxiliary decoders. Alongside a
language decoder that reconstructs text CoT, we introduce a visual world model decoder that predicts future-frame
tokens, forcing the latent space to internalize the causal dynamics of road geometry,
agent motion, and environmental change. A three-stage training pipeline progressively aligns these latents with trajectory,
language, and visual objectives, ensuring stable joint optimization. In inference, the auxiliary decoders are discarded, and all latent
tokens are prefilled in a single parallel pass, matching the speed of answer-only prediction. Across four
benchmarks, OneVL becomes the first latent CoT method to surpass explicit CoT, delivering superior accuracy at answer-only latency. These results show that with world model supervision, latent CoT produces more generalizable representations than verbose
token-by-
token reasoning. Code has been open-sourced to the community. Project Page: https://xiaomi-
embodied-intelligence.github.io/OneVL
Read the full abstract ▾
All authors Jinghui Lu, Jiayi Guan, Zhijian Huang, Jinlong Li, Guang Li, Lingdong Kong, Yingyan Li, Han Wang, Shaoqing Xu, Yuechen Luo, Fang Li, Chenxu Dang, Junli Wang, Tao Xu, Jing Wu, Jianhua Wu, Xiaoshuai Hao, Wen Zhang, Tianyi Jiang, Lingfeng Zhang, Lei Zhou, Yingbo Tang, Jie Wang, Yinfeng Gao, Xizhou Bu, Haochen Tian, Yihang Qiu, Feiyang Jia, Lin Liu, Yigu Ge, Hanbing Li, Yuannan Shen, Jianwei Cui, Hongwei Xie, Bing Wang, Haiyang Sun, Jingwei Zhao, Jiahui Huang, Pei Liu, Zeyu Zhu, Yuncheng Jiang, Zibin Guo, Chuhong Gong, Hanchao Leng, Kun Ma, Naiyan Wang, Guang Chen, Kuiyuan Yang, Hangjun Ye, Long Chen
Computation and Language · Cross-listed from cs.AI
WorldDB: A Vector Graph-of-Worlds Memory Engine with Ontology-Aware Write-Time Reconciliation
Harish Santhanalakshmi Ganesan
Persistent
memory is the bottleneck separating stateless chatbots from long-running agentic systems.
Retrieval-augmented generation (RAG) over flat vector stores fragments facts into chunks, loses cross-session identity, and has no first-class notion of supersession or contradiction. Recent bitemporal knowledge-graph systems (Graphiti, Memento, Hydra DB) add typed edges and valid-time metadata, but the graph itself remains flat: no recursive composition, no content-addressed invariants on nodes, and edge types carry no behavior beyond a label. We present WorldDB, a
memory engine built on three commitments: (i) every node is a world – a container with its own interior subgraph, ontology scope, and composed
embedding, recursive to arbitrary depth; (ii) nodes are content-addressed and immutable, so any edit produces a new hash at the node and every ancestor, giving a Merkle-style audit trail for free; (iii) edges are write-time programs – each edge type ships on_insert/on_delete/on_query_rewrite handlers (supersession closes validity, contradicts preserves both sides, same_as stages a merge proposal), so no raw append path exists. On LongMemEval-s (500 questions, ~115k-
token conversational stacks), WorldDB with Claude Opus 4.7 as answerer achieves 96.40% overall / 97.11% task-averaged accuracy, a +5.61pp improvement over the previously reported Hydra DB state-of-the-art (90.79%) and +11.20pp over Supermemory (85.20%), with perfect single-session-assistant recall and robust performance on temporal
reasoning (96.24%), knowledge update (98.72%), and preference synthesis (96.67%). Ablations show that the engine's graph layer – resolver-unified entities and typed refers_to edges – contributes +7.0pp task-averaged independently of the underlying answerer.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
ESsEN: Training Compact Discriminative Vision-Language Transformers in a Low-Resource Setting
Clayton Fields and Casey Kennington
Vision-
language modeling is rapidly increasing in popularity with an ever expanding list of available models. In most cases, these
vision-
language models have parameters in the tens of billions, which is necessary for some needs, but in many cases smaller models are necessary (e.g., on edge devices or independent robotic platforms). Unfortunately, there is little research in producing light-weight models or in training them with small
datasets. Inspired by the
language learning progression and data sparsity in child development, in this paper, we address both of these goals in a systematic fashion. We show that two-tower encoder models are superior to one-tower encoders in low-resource settings for discriminative English tasks. We show also that incorporating traditional convolutional networks into the two-tower
transformer architecture can help produce parameter efficient
vision-
language models. Finally, we show that the cross-modal fusion module of two-tower encoders can vary significantly in shape and size while producing the same results. In addition, we present ESsEN, a compact
vision-
language model that can be trained end-to-end with relatively few resources that performs as well on several tasks with only a fraction of the parameters compared to other models. The experimental results and the tools we present here make
vision-
language modeling more accessible to a wider variety of researchers.
Read the full abstract ▾
Computation and Language
BhashaSutra: A Task-Centric Unified Survey of Indian NLP Datasets, Corpora, and Resources
Raghvendra Kumar, Devankar Raj and Sriparna Saha
India's linguistic landscape, spanning 22 scheduled
languages and hundreds of marginalized dialects, has driven rapid growth in NLP
datasets,
benchmarks, and pretrained models. However, no dedicated survey consolidates resources developed specifically for Indian
languages. Existing reviews either focus on a few high-resource
languages or subsume Indian
languages within broader
multilingual settings, limiting coverage of low-resource and culturally diverse varieties. To address this gap, we present the first unified survey of Indian NLP resources, covering 200+
datasets, 50+
benchmarks, and 100+ models, tools, and systems across text,
speech,
multimodal, and culturally grounded tasks. We organize resources by linguistic phenomena, domains, and modalities; analyze trends in annotation, evaluation, and model design; and identify persistent challenges such as data sparsity, uneven
language coverage, script diversity, and limited cultural and domain generalization. This survey offers a consolidated foundation for equitable, culturally grounded, and scalable NLP research in the Indian linguistic ecosystem.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning
Hen Davidov, Nachshon Cohen, Oren Kalinsky et al. (+4 more)
LLMs utilizing chain-of-thought
reasoning often waste substantial compute by producing long, incorrect responses. Abstention can mitigate this by withholding outputs unlikely to be correct. While most abstention methods decide to withhold outputs before or after generation, dynamic mid-generation abstention considers early termination of unpromising
reasoning traces at each
token position. Prior work has explored empirical variants of this idea, but principled guidance for the abstention rule remains lacking. We present a formal analysis of dynamic abstention for LLMs, modeling abstention as an explicit action within a regularized reinforcement learning framework. An abstention reward parameter controls the trade-off between compute and information. We show that abstaining when the value function falls below this reward strictly outperforms natural baselines under general conditions. We further derive a principled and efficient method to approximate the value function. Empirical results on mathematical
reasoning and toxicity avoidance tasks support our theory and demonstrate improved selective accuracy over existing methods.
Read the full abstract ▾
All authors Hen Davidov, Nachshon Cohen, Oren Kalinsky, Yaron Fairstein, Guy Kushilevitz, Ram Yazdi, Patrick Rebeschini
Computation and Language
CAPO: Critic-Guided Action-Aligned Policy Optimization for Advancing LLM Agent Capabilities
Daoyu Wang, Qingchuan Li, Mingyue Cheng et al. (+6 more)
Reinforcement learning (RL) has become a key technique for improving the agentic capabilities of large
language models (LLMs). Although critic-free methods such as GRPO are increasingly popular, we argue that critic-based methods remain well suited to long-horizon agentic tasks because their critic models can assess each state and assign credit to different decisions. However, representative critic-based methods such as PPO still organize value estimation, credit assignment, and policy updates at
token granularity, whereas the environment state changes only after the
agent completes a full action. In this work, we propose CAPO, a critic-guided action-aligned policy optimization method for advancing LLM
agent capabilities. CAPO introduces two action-aligned designs. First, it estimates the state value at the action boundary using a critic model and assigns the corresponding advantage to the complete action rather than individual
tokens. Second, CAPO introduces a novel action-aware policy ratio that aggregates
token ratios within each action with length calibration for consistent updates across actions of different lengths. These two designs effectively achieve fine-grained credit assignment and consistent policy updates while remaining compatible with
token-level gradient computation. Experiments across multi-hop QA, academic paper search, and text-world action tasks show that CAPO consistently outperforms representative RL methods. Further analyses validate both designs and reveal how action-aligned optimization improves multi-turn RL training. We hope this work provides a practical path for training more capable LLM
agents.
Read the full abstract ▾
All authors Daoyu Wang, Qingchuan Li, Mingyue Cheng, Jie Ouyang, Shuo Yu, Chunli Liu, Shijin Wang, Qi Liu, Enhong Chen