Computation and Language
Selection Is Retrieval, Abstention Is Not: On-Device Tool Routing over 70 Korean-English Actions
Janghoon Lee
An AI assistant that calls tools makes two decisions on every request: which tool to invoke, and whether any available tool applies. In the usual design a single
language model makes both, by emitting a call or by declining to emit one. On a device that has to answer without a server, the
language model is what makes that design expensive, dominating both the latency and the
memory of the router. The common alternative is to remove the model completely and rank the catalog of local actions with a retriever instead. That substitution is not symmetric across the two decisions. A retriever returns its highest-scoring candidate for every input and cannot signal that the catalog holds no valid action. Our earlier study found that constraining a decoder to a tool grammar repairs malformed output without improving the choice. What the substitution costs in each decision has not been measured. We evaluate the two decisions separately over 600 Korean and English requests and a catalog of 70 local actions. The router may also ask for a missing slot, reply, or delegate. Half the in-catalog requests reuse catalog vocabulary and half paraphrase it, separating lexical overlap from the action requested. Character 3-gram BM25 selects 162 of 164 lexically matched requests and 85 of 166 paraphrases. Restricting the candidate set to seven raises the paraphrase figure to a mean of 0.825 over five trials. No classifier over its score features separates in-catalog from out-of-catalog above 0.697 area under the curve, where the frozen encoder multilingual-e5-base reaches 0.806. Using that encoder for abstention alone keeps 376 of the requests local and misroutes 9 of the 150 needing delegation. Abstention, not selection, is where a neural component is required. A neural ranker improves every quality metric and is rejected on latency and
memory rather than accuracy.
Read the full abstract ▾
Computation and Language
DyMT-ESB: Dynamic Multi-Turn Evaluation of Social Bias in User-LLM Interactions
Rem Hida, Masahiro Kaneko, Daisuke Oba et al. (+2 more)
Warning: This paper contains examples of stereotypes and social bias. LLMs are increasingly used in interactive settings by the general public, making the evaluation of model behavior in multi-turn conversational scenarios important for safety, including stereotyping-related harms. However, existing multi-turn social bias evaluations often rely on pre-specified or template-based user inputs that do not adapt to model responses and typically assume a fixed dialogue length in advance. In this paper, we study social bias dynamics in response-conditioned multi-turn interactions using a controlled evaluation protocol that generates follow-up user queries from the evolving dialogue history and allows evaluation over variable numbers of turns. Experimental results show that LLMs exhibit social bias even in coherent, response-conditioned multi-turn interactions, revealing late-emerging bias, non-monotonic bias patterns, and bias re-emergence. These results motivate evaluations that extend beyond fixed-turn, pre-scripted protocols. Our findings highlight the importance of analyzing social bias as a turn-level dynamic phenomenon.
Read the full abstract ▾
All authors Rem Hida, Masahiro Kaneko, Daisuke Oba, Danushka Bollegala, Naoaki Okazaki
Computation and Language
Fallacy Benchmarks Measure Scheme Recognition, Not Fallacy Detection
Navyansh Singh, Animesh Pathak and Aarav Singh
Fallacy-detection
benchmarks pair fallacy classes with a single "valid" or "none" class that takes everything data collection did not label as a fallacy. A detector has two jobs, deciding whether an argument is fallacious and naming which fallacy it commits, and the false-positive rate is meant to measure the first. We show that what these
benchmarks actually score is scheme recognition, the ability behind the second job. Their own test sets already show it: when a classifier misses a fallacy, the error lands on "none" rather than on another fallacy type, so detection is failing while classification holds. The reason is what the valid class lacks. The negatives that separate the two jobs are correct arguments using the same argumentation scheme as a fallacy, and they are scarce: nearly absent from the four
benchmarks we examined, and rare even under deliberate search. A detector is therefore never tested where recognizing a scheme and judging its use come apart, and can pass on recognition alone. We construct the missing arguments, together with a control condition from the same pipeline that differs only in scheme, so whatever generation contributes, it contributes to both. The classifier labels the scheme-matched negatives as the source fallacy, and labels the wrong-scheme negatives as the scheme they actually use 85.9% of the time and as the source type 0.4%. The classifier has learned which scheme an argument uses, not whether it uses it correctly. The over-flagging follows: a model that scores 16.6% on CoCoLoFa's own valid class flags 58.9% of the constructed arguments. The same dissociation appears in three zero-shot LLM detectors that never saw these
benchmarks. We release the items as Scheme Foils. A reported false-positive rate should not be trusted as a measure of detection until the valid class has been audited for scheme-matched coverage.
Read the full abstract ▾
Computation and Language
STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolution
Yajie Yu, Mark Lee and Yue Feng
Large
language models (LLMs) often suffer from capability stagnation in self-improvement training because fixed difficulty levels fail to adapt to their evolving proficiency. To address this issue, we propose STRETCH (Self-Taught
Reasoning Evolution via Targeted CHallenge), a unified framework inspired by cognitive scaffolding theory. STRETCH introduces a dynamic Stretch Zone mechanism that continuously aligns question difficulty with the model's solving capability. Within a single parameter space, the model alternates between a Scaffolder that generates adaptive, boundary-pushing challenges and a Learner that that optimizes its solving trajectories through reinforcement learning. This dual-loop co-
evolution effectively stabilizes training, mitigates reward hacking and promote progressive
reasoning growth. Experiments on both negotiation and operation research
benchmarks demonstrate that STRETCH consistently outperforms strong prompting and domain-specific baselines. Further scaffolder configuration analysis shows that dynamic difficulty
alignment is critical for sustained capability improvement and synchronized
reasoning evolution.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence
Sebastian Gerstner, Hilal AlQuabeh, Kentaro Inui et al. (+1 more)
We analyze the learned input-output behavior of GLU-based
neurons in large
language models (LLMs). We propose a simple analysis method: For each
neuron, we compute the cosine similarities between its input (reading) and output (writing) weight vectors. In this scheme, a strong negative cosine similarity indicates the
neuron weakens the direction it detects in the residual stream, so we call this a weakening
neuron. This allows us to gain a number of novel insights. First, we show that nine different LLMs have similar patterns: weakening
neurons appear mostly in late layers whereas their counterparts, (conditional) strengthening
neurons, are frequent in early-middle layers. Second, we find that weakening
neurons display surprising behavior: even though there are few, they activate often and have a large influence on model behavior. Third, weakening
neurons have a strong effect on model output when gate values are negative – which is surprising since negative gate values are not expected to encode functionality.
Read the full abstract ▾
All authors Sebastian Gerstner, Hilal AlQuabeh, Kentaro Inui, Hinrich Schütze
Computation and Language
PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?
Mika Okamoto and Ansel Kaplan Erol
As corporate AI adoption continues to grow, enterprise-grade LLM
agents are being deployed into sensitive contexts such as hiring, healthcare, and finance. In these contexts, compliance with rules specified in an
agent's system context is a first-order legal concern. Currently, no evaluation framework systematically measures which LLM models tend to violate compliance rules, especially under pressure from a persistent user, a hurried manager, or circumstances where violation is convenient or attractive. We introduce PACT (Pressure-Applied Compliance Testing), a
benchmark for rule-following under pressure in AI
agents assisting employees in daily tasks across twelve regulated enterprise domains and forty-eight scenarios, each set in a realistic multi-turn conversation. Each
benchmark item pairs a standing rule against a rule-violating shortcut, and applies a battery of pressures across different wordings and system-
prompt modes. We construct PACT component by component under strict LLM-as-judge auditing to ensure samples are unambiguous, ungameable, and realistic enough to avoid eliciting evaluation-aware behavior. We use PACT to profile LLM compliance across six complementary metrics that create a holistic picture of an AI assistant's robustness under pressure and throughout multi-turn conversations, its transparency, and ability to correctly discern where a rule applies. We aggregate this profile into PACTScore, a reliability-weighted compliance rate over all items and modes. Our results across 22 common LLM models spanning multiple providers and sizes show substantial variability in compliance across models and metric dimensions. Even the strongest assistants mis-apply a rule on 6 to 10% of items, and ordinary user pressure raises the violation rate by 65% on average. PACT highlights compliance risks in LLM assistants, motivating guardrails and careful model selection.
Read the full abstract ▾
Computation and Language · Cross-listed from quant-ph
Julian Hager, Michael Kölle, Gerhard Stenzel et al. (+3 more)
We propose a compact NISQ-compatible
quantum transformer architecture for synthetic QNLP sequence modelling. The model preserves the autoregressive next-
token interface of a classical
transformer, but replaces
attention and feed-forward sublayers with variational
quantum encoder blocks, connector
circuits, decoder blocks and a direct two-
qubit measurement readout.
Token contexts are angle-encoded into small
quantum registers, processed by parallel variational heads and encoder integration
circuits and conditioned through decoder ancillae to produce a distribution over a four-
token vocabulary. We evaluate several architecture variants on deterministic and lexicographic grammar-generation tasks against a compact classical
transformer baseline. The
quantum models are trainable end-to-end and learn nontrivial grammar structure, including perfect deterministic generation in individual runs and high lexicographic validity in the strongest variant. The classical baseline remains more accurate and stable and the
quantum models are sensitive to initialization. The contribution is therefore not a claim of
quantum advantage, but a concrete architecture and evaluation of
transformer-inspired QNLP sequence modelling under near-term
quantum constraints.
Read the full abstract ▾
All authors Julian Hager, Michael Kölle, Gerhard Stenzel, Tobias Rohe, Jonas Stein, Claudia Linnhoff-Popien
Computation and Language
A Probe Shift Is Not a Fairness Fix: The Limits of Representation Steering in Speech Models
Nicolas Bourrel, Abderrahmane Issam and Gerasimos Spanakis
Automatic
speech recognition (ASR) systems exhibit unequal error rates across speaker groups, motivating interventions on their internal representations. We ask whether speaker-linked attributes that are linearly readable from pretrained ASR encoders yield useful directions for reducing group word-error-rate (WER) gaps. Across Whisper-medium, HuBERT-large, and Wav2Vec2-large on Common Voice and the
Speech Accent Archive, we probe every encoder layer for metadata-derived sex/gender, age, and native/accent labels; construct centroid and probe-derived directions; inject them at selected layers; and compare downstream probe trajectories with matched WER changes. Sex labels are highly decodable (best macro-F1 0.924–0.941), native/accent labels are also above chance (0.544–0.696), and age is weaker (0.354–0.397). Of 22 post-selected reruns, nine have 95% paired-bootstrap intervals entirely below zero, yet every absolute source-group WER reduction is below 0.7 percentage points. Conversely, a local target-class probe rate can rise from 8.09% to 99.87% while WER worsens. Linear readability is therefore neither evidence of causal use nor a reliable mitigation method. Our results motivate evaluating
speech-bias interventions jointly at representation, propagation, and task levels.
Read the full abstract ▾
Computation and Language
Machine Translation between English and Syriac (East Syriac Dialect) using Statistical Machine Learning
Hadiana Sliwa and Hossein Hassani
UNESCO considers the Assyrian (Syriac)
language an endangered
language. Although Assyrians speak the
language worldwide, the speaking population is uncertain (ranging from 500,000 to 1,500,000). Syriac is also one of the least studied
languages in Natural
Language Processing (NLP). Despite advances in Machine Translation (MT) over the past decade, the lack of publicly available corpora and the orthographic complexity of the Syriac script, specifically the Madnkhaya script, have left this
language entirely ignored in the computational linguistics literature. This study develops the first phrase-based Statistical MT (SMT) model for English-to-Assyrian MT using the Moses framework. We created a
dataset of 38,847 sentence pairs from the complete English and Syriac Bible, merging a pre-existing New Testament
dataset with an Old Testament built from scratch through PDF extraction, using custom segmentation scripts and manual
alignment review by three bilingual annotators. The Syriac side of the corpus undergoes diacritic removal and Byte-Pair Encoding tokenization to reduce orthographic sparsity before training. We trained and evaluated six models using different configurations and splitting-scheme ratios,
language model order, distortion limits, and the inclusion of an Operation Sequence Model. The best-performing configuration achieves a word-level BLEU score of 23.54. Human evaluation by 11 native Assyrian speakers resulted in mean adequacy and fluency scores of 3.42 and 3.34 out of 5, respectively. These results are consistent with comparable low-resource SMT models trained on Biblical corpora for morphologically rich Semitic
languages. The corpora, scripts, and trained model are publicly available, providing the research community with the first systematically curated English-Syriac
dataset and a reproducible baseline for future MT and broader NLP work on this endangered
language.
Read the full abstract ▾
Computation and Language
Align, Integrate, and Fire: Efficient Token-Level Alignment for Zero-Shot SpeechLLMs
Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes et al. (+1 more)
While Large
Language Models excel in natural
language processing, efficiently extending their capabilities to spoken input remains a significant challenge. Existing methods for building SpeechLLMs often rely on computationally expensive full-model fine-tuning, or employ parameter-efficient projectors that suffer from inefficient
token sequence lengths and costly full-model supervision. In this paper, we introduce Aligned Continuous Integrate-and-Fire, a highly efficient framework for zero-shot
speech processing. Our method dynamically compresses continuous acoustic frames into the exact discrete
token length of the target text utilizing explicit Dynamic Time Warping
alignments. This allows our initial training stage to establish a robust acoustic-to-semantic bridge using lightweight distance metrics, entirely bypassing the computationally expensive LLM forward pass. For subsequent fine-tuning, we propose a
memory-efficient knowledge
distillation objective that targets a single LLM layer, performing competitively with full-model cross-
entropy training at a fraction of the computational cost. Through extensive evaluations on Automatic
Speech Recognition and
Speech Translation, we demonstrate that our method achieves superior performance compared to prior parameter-efficient baselines.
Read the full abstract ▾
All authors Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
Computation and Language
Size Matters: Foundation Model for Czech HTML documents
Martin Dvořák, Vít Tlustoš, Artyom Voronin et al. (+4 more)
Creating universal, high-quality representations of web documents in high-traffic industrial environments requires models that are both performant and economic. Existing approaches, however, often depend on large models, overlook the structural information inherent in HTML, or are constrained by short context windows, limiting their ability to process real-world web pages. We present HTML-LM, a compact foundation model with 154 million parameters that addresses these limitations through HTML-aware training and a ModernBERT-based architecture. It was trained on 100 million web documents using multiple objectives, including masked
language modeling, bag-of-words prediction, and contrastive
distillation from large
language models. Consequently, HTML-LM sets a new state-of-the-art for classification and regression applications in the Czech Internet domain, surpassing both larger encoders and small-sized LLMs. The model is deployed in production, processing thousands of web documents per second, and released to the community under the CC BY-NC 4.0. https://huggingface.co/Seznam/html-lm.
Read the full abstract ▾
All authors Martin Dvořák, Vít Tlustoš, Artyom Voronin, Martin Habrovec, Kateřina Podlesná, Barbora Rišová, Josef Vonášek
Computation and Language · Cross-listed from cs.RO
ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models
Shijie Lian, Bin Yu, Zhaolong Shen et al. (+5 more)
Action tokenizers play a central role in autoregressive
vision-
language-action (VLA) models, determining both the targets for policy training and the executable commands recovered from predicted
tokens. Their fidelity is commonly evaluated using pointwise reconstruction metrics such as mean squared error (MSE), yet small individual errors do not fully characterize how faithfully action adjustments across demonstrations are preserved. After compression, similar actions may still cluster around a representative motion, while the adjustments needed for different contexts are diminished, distorted, or even reversed. We introduce physical rank consistency (PRC) to measure how well tokenization preserves local physical distance rankings after reconstruction. Evaluating decoded actions provides a common reference across
token vocabularies and decoder architectures, complementing pointwise accuracy with a measure of relational fidelity. We further present ActionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization. Physical rank preservation supervises near-far ordering in encoder and quantized feature distances, while quantization regularization applies the same ordering to codeword assignment distributions. Both objectives augment reconstruction, producing discrete action
tokens for standard autoregressive policy learning and execution through a frozen decoder. Under the same Qwen3-VL-4B policy training setup, ActionPiece achieves 94.8% on LIBERO and 68.8% on unseen LIBERO-Plus, with additional evaluations reaching 71.9% on SimplerEnv and 51.5% across VLA-Arena L0-L2. Component ablations show that the two objectives jointly improve PRC and policy success, demonstrating the value of physical relationship supervision for action tokenization.
Read the full abstract ▾
All authors Shijie Lian, Bin Yu, Zhaolong Shen, Xiaopeng Lin, Yichao Du, Zhirui Zhang, Laurence T. Yang, Kai Chen
Computation and Language · Cross-listed from cs.MM
Divide and Conquer: Mixture-of-Bottleneck Experts in Informative Ordinal Space for Video-based Multimodal Sentiment Analysis
Ronghao Lin, Qiaolin He, Zefeng Lu et al. (+5 more)
Video-based
Multimodal sentiment analysis (MSA) must handle information from text, audio, and image sequence in human speaking
videos, yet current methods often fail to integrate modalities with task awareness. Most models treat
video sentiment prediction as a single task, overlooking its ordinal nature, and their fusion strategies struggle to capture diverse unique and synergic cues across modalities. To address these limitations, we adopt a divide-and-conquer perspective by reformulating MSA as an ordinal regression problem and decoupling it into polarity recognition and intensity prediction. Driven by information theory, we introduce a Mixture-of-Bottleneck (MoB) framework that assigns different latents to polarity- and intensity-specific experts for different modalities. With the learning of information bottleneck, each expert learns compact and task-relevant representations while filtering out redundancy and noise. A
multimodal bottleneck routing fusion module then fuses these expert latents with hard mining strategy, guiding the prediction in the ordinal sentiment space. Extensive experiments on 4 MSA
datasets and 4
language models show that MoB effectively leverages informative latents from diverse modalities and captures general sentiment structure. Beyond stronger performance, MoB comprehensively captures fine-grained intra- and inter-modal dynamics, enabling more trustworthy localization of nuanced
video sentiment signals.
Read the full abstract ▾
All authors Ronghao Lin, Qiaolin He, Zefeng Lu, Yichu Liu, Li Huang, Sijie Mai, Haifeng Hu, Yap-peng Tan
Computation and Language · Cross-listed from cs.AI
Disentangling Long-Term Memory via Latent Neuro-Symbolic Reasoning
Cai Ke, Xinghao Chen, Xiaoyu Shen et al. (+6 more)
Personalized
agents are required to reason over long-term history interactions to infer both explicit preferences and implicit behavioral evidence. While early flat
retrieval methods score
memory fragments independently and neglect the distributed information, current structured
memory frameworks rely on query-agnostic static graphs that fail to capture the context-dependent relations. Crucially, raw textual
memories are inherently entangled and noisy, making fine-grained personalization and cross-session
reasoning computationally prohibitive. To this end, we present LGM, a novel neuro-symbolic framework that shifts long-term
memory disentanglement into a continuous latent space. Specifically, (i) instead of persisting fixed graphs, we design a tailored latent graph construction with a sparse autoencoder. Subject to each query, it maps historical interactions into latent
memory nodes and disentangles the
memory traces into sparse concept activations, dynamically synthesizing query-aware relational edge weights. (ii) A graph encoder then treats the query
embedding as a conditioning preference to direct non-linear message passing across the task-specific latent subgraph. This yields a highly expressive
memory representation for effective activations. Extensive experiments on long-term personalization
benchmarks demonstrate that LGM significantly outperforms state-of-the-art baselines in capturing both explicit and implicit preferences while enabling personalized responses.
Read the full abstract ▾
All authors Cai Ke, Xinghao Chen, Xiaoyu Shen, Keyu Chen, Siyu An, Junnan Dong, Ruifeng Xu, Ruizhi Qiao, Xing Sun
Computation and Language
M-SQE: Multilingual Skill Quality Estimation for Enhancing Language Equality in Agentic Skill Use
Yilun Liu, Shimin Tao, Minggui He et al. (+9 more)
Agent skills, reusable procedural documents that extend LLM
agents beyond their parametric
memory, have become an important interface for deploying
agents on real-world tasks. Community-maintained skill libraries built around this interface are growing rapidly. However, this ecosystem remains deeply English-centric: our audit finds that low-resource
languages such as Swahili and Hindi have no in-
language skill content, so
retrieval often returns a skill written in a different
language than the query, degrading accuracy and recall. A practical solution is to synthesize in-
language skills for
retrieval but the quality can be unreliable, so relevance in this setting alone often surfaces a related but unusable candidate. To address this, we propose M-SQE, a post-
retrieval Multilingual Skill Quality Estimation framework that scores candidates via a Theory view for intrinsic quality and an Action view for task-grounded utility, unified into a domain-conditioned final score. We evaluate M-SQE across three skill-use domains: general, tool-use, and cultural tasks. Empirically, we build three-layer candidate skill pools mirroring today's ecosystem, where M-SQE's task success exceeds existing baseline's average by at least +3.5 points across three different retrievers. Particularly, M-SQE lifts the lowest-resource
languages most (+12.9pp on Hindi and +5.6pp on Swahili) and achieves strong performance across all six culture regions, thereby moving agentic skill use toward linguistic and cultural equality.
Read the full abstract ▾
All authors Yilun Liu, Shimin Tao, Minggui He, Chenxin Liu, Li Zhang, Chen Liu, Miao Zhang, Jiaxin Guo, Min Zhang, Liqun Deng, Xiaojun Meng, Daimeng Wei
Computation and Language
Planning or Improvisation? Stress-Testing the Poetry Planning Site on Open Models and Open Cross-Layer Transcoders
Éric Jacopin
Lindsey et al. (2025) report that Claude 3.5 Haiku plans rhymes: features for candidate rhyme words are active on the newline before a line is written, and a suppress-and-inject intervention redirects the line only when applied there (their Figure 13). We test how far this generalizes on seven
cells crossing four open models (0.6B to 2.6B parameters) with six open cross-layer transcoders (CLTs), on one consumer GPU, decomposing the claim into position specificity (C1), newline site identity (C2), and a newline-resident plan (C3). This is a stress test rather than a faithful reproduction: attribution graphs are unavailable for these CLTs, so features are found bottom-up from decoder vectors. C1 generalizes, in every
cell and in all 247 of 444
prompt-by-inject pairs with a detectable effect, but the effective position is the final
prompt token, adjacent to emission, and only two
cells reach behaviorally meaningful probabilities. C2 and C3 are not recovered by any probe: a census of every active feature finds no rhyme-anticipating enrichment at the newline, and steering the newline while the model composes the whole line, over 36 runs and 8,640 sampled lines, shows why. That intervention is strong but one
token long, making the injected word the first word of the composed line in 703 of 720 samples and leaving the rhyme six words later untouched. A final test drops the transcoder entirely: patching the newline's whole residual, at every layer, from a minimal-pair poem whose third line ends on a different rhyme moves the rhyme in 11 of 1,260 composed lines against 4 at baseline, with a design resolving 1.4%. We read this as a boundary condition rather than a refutation: at this scale and with these transcoders, the causal site is emission-adjacent. We reproduce Figure 13's shape, not its mechanism. Code and data are public (code: github.com/PCfVW/poetry-planning-site).
Read the full abstract ▾
Computation and Language
Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning
Zhuo Chen, Zhen Zhang, Xinyu Wang et al. (+1 more)
Multi-turn
agent trajectories often contain redundant rounds (failed tool calls, parallel sub-queries, verification-only steps) that inflate both training and inference cost. We propose viewing each trajectory as a round-level dependency DAG that exposes which rounds are globally load-bearing for the final answer, and fine-tune
agents on trajectories refined through this DAG. Given an LLM-annotated DAG, these edits are deterministic and interpretable, with optional rephrasing. Models trained on these refined trajectories consistently outperform those trained on the original trajectories at lower inference cost. Specifically, across four multi-modal QA
benchmarks, our refinements improve downstream accuracy by up to $1.7$\,pp over vanilla SFT (and $5.7$\,pp over an LLM-deletion baseline) while reducing per-sample inference messages by up to approximately $40\%$ and inference
tokens by up to approximately $48\%$, translating to substantial savings in compute and serving cost. Code is available.
Read the full abstract ▾
All authors Zhuo Chen, Zhen Zhang, Xinyu Wang, Kewei Tu
Computation and Language
Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts
Christopher Bagdon, Carina Silberer and Roman Klinger
Emotions are an essential aspect of human communication, particularly on social media, where authors frequently combine text and images to convey their emotions. Yet prior work on emotion analysis of social media posts has overlooked two important aspects in regard to measuring how well readers can reconstruct the authors' intent: (1)~the image modality, with most work focusing solely on text, and (2)~the real-world events that trigger the expressed emotions, and their relationship to the post content. We therefore study the relation between (a) the author's experience of the event that caused them to write a social media post and (b) the content of the post, with a focus on readers' capability to reconstruct that emotion expression. To do that, we introduce the
Multimodal Multi-Emotion-Model
dataset Mult2EMo, created by collecting annotations from both authors and readers on the posts and their triggering events. We find that reconstruction is possible but challenging for both human readers and computational models. We show that understanding the triggering event is crucial for accurate reconstruction, and that reconstruction is particularly challenging when posts rely heavily on the image to express emotion.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Market Signal Injection: Adversarial Context Manipulation of LLM Pricing Agents
Dohun Lee and Hyunwoo Park
Large
language model (LLM) pricing
agents may respond to how
market data is presented, even when its numerical values remain unchanged. We introduce
market signal injection (MSI), an attack that manipulates numerical formatting, competitor ordering, or qualitative
market commentary without issuing explicit instructions. We evaluate nine open-weight models in simulated Bertrand duopoly and triopoly
markets and three proprietary models in duopoly
markets. Sentiment-based attacks produce the largest behavioral shifts, which propagate to other firms and alter profits and consumer surplus. Susceptibility varies across model families, and larger models are not consistently more robust. Matched neutral-text controls and a rule-based
agent support a framing-based account of these shifts under the fixed demand parameters of our simulation. Episode-held-out probes distinguish baseline from attacked activations in all eleven re-evaluated model–condition pairs: linear AUC is 1.00 and MLP AUC ranges from 0.93 to 0.99. This separability does not by itself identify harmful pricing decisions. Input canonicalization removes the tested sentiment attacks, while decision boundary anchoring, which combines
prompt constraints with output projection, provides partial mitigation under the tested adaptive attacks. These results identify data presentation as an attack surface for LLM pricing
agents and motivate defenses that account for interactions among
agents.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Faithful yet Collusive: Why Chain-of-Thought Monitoring Cannot Detect Collusion in LLM Pricing Agents under Oligopolistic Competition
Dohun Lee and Hyunwoo Park
Large
language models (LLM) deployed as autonomous pricing
agents may sustain supracompetitive prices through tacit coordination. We develop a causal graph divergence framework that separately measures structural faithfulness and intent faithfulness of LLM pricing
agents in Bertrand competition. Across nine LLMs under duopoly and triopoly conditions, collusive behavior and chain-of-thought (CoT) faithfulness dissociate along both dimensions: the most collusive model accurately reports cooperative intent yet reasons structurally unfaithfully, while the most structurally faithful model sustains supra-Nash pricing under both
market structures. These findings establish that CoT monitoring alone cannot serve as a standalone safeguard against algorithmic collusion.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CY
Understanding AI Provider Recommendations in Local Service Markets
Hazem Ibrahim and Yasir Zaki
When someone asks an AI assistant which doctor to see or which firm to trust with their savings, the answer is a referral. We audit AI provider recommendations in four registry-backed service domains across the 100 largest U.S. metropolitan areas, matching every recommendation against the official registry for its domain (Medicare clinician and facility records, and SEC adviser disclosures), under three conditions: an open-weight model, a proprietary model without web search, and the same proprietary model with search. Without search, both models largely fabricate recommendations in the domains the web covers thinly. Only 4% of the open-weight model's recommended doctors and 11% of the proprietary model's match a clinician in the queried city, and the open-weight matches are name coincidences: its matched clinicians are no likelier to be primary-care doctors than names drawn at random from the registry. With search, 64-71% of recommendations in the same domains match a real provider. Search also changes who is recommended. Without it, recommended advisory firms carry SEC misconduct disclosures at 3.6 times the registry base rate, even after adjusting for firm size; with search, significantly below it. Restaurants, where quality and visibility are separately measurable, show a 3-5x review-count premium but a rating premium of at most a tenth of a
star. Finally, search largely removes the metro-size penalty: without it, real recommendations concentrate in the largest metros; with it, match rates are similar across metro-size terciles. Whether an AI referral is trustworthy depends strongly on its
retrieval configuration rather than on the underlying model alone, yet an answer produced without
retrieval often carries no sign that its recommendations were never verified.
Read the full abstract ▾
Computation and Language
Shardul P. More and Tanuja S. Pawar
Large
Language Models (LLMs) frequently exhibit
hallucinations, presenting a major barrier to reliability in complex
reasoning tasks. While traditional detection methods rely on output-based confidence metrics, these logits are often miscalibrated by modern
alignment techniques. In this paper, we investigate the temporal
volatility of internal
attention mechanisms as an alternative diagnostic signal for
hallucination that does not depend on output calibration. By introducing an unsupervised metric for
attention dispersion, we show that epistemic uncertainty leaves a measurable trace within intermediate layers, where spikes in
attention entropy are associated with
reasoning breakdowns. We evaluate our approach on mathematical
reasoning benchmarks (GSM8K and MATH-500) using the Qwen2.5 model family (1.5B and 3B parameters), finding statistically significant AUC improvements of up to +0.076 over output-based baselines across all tested conditions. These findings suggest that
attention dispersion is a promising complement to traditional
hallucination detection methods, requiring further investigation across broader model families and task domains.
Read the full abstract ▾
Computation and Language
Knowledge-Graph Based Augmentation versus Retrieval Augmented Generation for Cultural-Related Question Answering
Pablo Poulenard, Yannis Karmim and Valentin Barrière
Large
language models (LLMs) suffer from a long-tail deficit: culturally specific facts, particularly those concerning underrepresented regions such as Latin America, appear too rarely in pretraining corpora to be reliably memorized.
Retrieval-Augmented Generation (RAG) addresses this by grounding generation in external text, but structured alternatives such as Knowledge Graphs (KGs) offer tighter control over what enters the context, along with potential gains in explainability and updatability. We
benchmark Graph-RAG against standard RAG on LatamQA, a culturally grounded multiple-choice
dataset spanning eight thematic categories. The graphs are built end-to-end from Wikipedia articles with KGGen, a recent open-domain extractor, without manual curation in our main setting. G-Retriever is competitive with RAG and reduces the error of the base LLM by 72% with a standard KG and 78% with a
benchmark-aware variant, the gap to RAG narrowing further as the graph is oriented toward task-relevant content. The trained projection transfers zero-shot to Portuguese without target-
language fine-tuning, indicating multilingual reach.
Read the full abstract ▾
Computation and Language
SEA-LION-v4.8: A Technical Report
Adila Aulia, Ahmed Dabeer, Ahn Jeongmi et al. (+38 more)
We introduce Nemotron-SEA-LION-v4.8, a family of Southeast Asian
Languages In One Network (SEA-LION) models built upon NVIDIA Nemotron 3. The family includes 30B-A3B and 120B-A12B models, with both continued-pretrained base checkpoints and post-trained variants. We adapt the models using Southeast Asian,
reasoning, code, and multilingual parallel data, followed by post-training with supervised fine-tuning and online on-policy
distillation. On SEA-HELM, the 30B-A3B model improves the overall SEA score from 46.06 to 51.57, while the 120B-A12B model improves from 49.30 to 63.44. Across seven Southeast Asian
languages, we observe broad capability gains with the 120B-A12B model showing broader and more consistent improvements across tasks.
Read the full abstract ▾
All authors Adila Aulia, Ahmed Dabeer, Ahn Jeongmi, Antonyrex Sajeban, Chan Hok Teng Adwin, Cheng Zi Yi Nicholas, Choa Hsueh Mei Esther, Heng Jonathan, Jann Railey Estrada Montalan, Lee Chwan Ren, Leong Wai Yi, Leong Wei Qi, Liew Rachel, Limkonchotiwat Peerat, Muhammad Ridzuan Bin Mokhtar, Nagarajan Karthik, Ng Boon Cheong Raymond, Ngee Chia Tai, Ngui Jian Gang, Nguyen Thanh Ngan, Ong Tat-Wee David, Pereira Mark, Phang Shi Wei Benjamin, Poon Joseph, Rengarajan Hamsawardhini, Susanto Yosephine, Sutaveephamochanon Anocha, Tan Choon Meng, Tan Chor Phin Evelyn, Tan Le Min Sheryl, Tan Siao Wei Jessica, Tan Yixian, Tasawong Panuthep, Tee Jun Yun, Teng Kok Wai Walter, Teo Eng Sipp Leslie, Tjhi William, Tuchinda Pume, Wu Donghang, Yong Xianbin, Zhang Zhou
Computation and Language
Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agents
Yi Yu, Liuyi Yao, Yaliang Li et al. (+2 more)
Large
language model (LLM)
agents increasingly tackle long-horizon tasks through multi-step environment interaction, yet a single erroneous action can alter subsequent states and observations, causing errors to compound over time. Existing methods either correct the context without repairing altered environment states or restore earlier states while discarding useful experience, making it difficult to both eliminate failure conditions and avoid repeating past mistakes. We argue that reliable recovery should instead be treated as a rollback-boundary control problem that jointly determines when to intervene, where to resume, and what information should survive recovery. Based on this view, we propose Rollback-Induced Reflection (RIR), a unified recovery framework that restores execution to a selected prior state while carrying forward reusable knowledge distilled from the abandoned trajectory to guide subsequent decisions. We further characterize recovery through a unified operator over rollback depth and retained
memory, providing a general view of state restoration and knowledge retention. Experiments on three long-horizon
benchmarks show that RIR consistently improves average task performance across multiple LLM backbones, with structured reflection
memory preserving useful experience and selective rollback enabling efficient recovery.
Read the full abstract ▾
All authors Yi Yu, Liuyi Yao, Yaliang Li, Enshu Wang, Libing Wu
Computation and Language · Cross-listed from cs.SE
Relationally Guided Use Case Modeling with LLMs
Guangyu Wang, Bangqi Li, Ji Wu et al. (+1 more)
Use case flows are important elements of use case modeling because they support downstream software engineering activities, including requirements analysis, architectural and detailed design, and test case generation. However, constructing them manually is costly and expertise-intensive, while existing automated approaches still struggle to preserve semantic consistency, control-flow logic, data-flow logic, and the intended system boundary, especially when identifying branch points and generating alternative flows. To address this problem, we propose FlowGen for complete use case flow construction. FlowGen uses LLM-based Semantic Information Processing (SIP) to extract semantic elements, constructs a Semantic Relational Graph (SRG) encoded by an enhanced R-GAT for basic flow generation (BFGen), and further supports branch point prediction through BPP and branch-conditioned alternative flow generation through AFGen. Evaluations on 13 public and 7 industrial
datasets show that FlowGen consistently outperforms competitive baselines in all three core components. In particular, BFGen improves over the best baseline by 14% in Precision, 7-25% in Recall, 11-30% in F1, and 10-19% in AUC; BPP improves Precision by 30-110%, Recall by 33-91%, and F1 by 32-117%; AFGen improves Precision by 8-23%, F1 by 5-18%, and AUC by 0.6-2.5%. Moreover, we validate the effectiveness of the LLM-based SIP module and the
attention preservation factor in BFGen, analyze the impact of requirement completeness on BFGen, and examine how different scopes of branch-related context affect AFGen.
Read the full abstract ▾
All authors Guangyu Wang, Bangqi Li, Ji Wu, Zhijun Shao
Computation and Language
Made in Hungary: Comments on the performance of generative language models
Mátyás Osváth, Enikő Héja and Noémi Ligeti-Nagy
In recent years, three initiatives have emerged to develop generative
language models in Hungary. The motivation behind them is the same. For Hungarian, no model with the given capability existed, or existing English-centric models offered limited proficiency. A detailed examination of the corresponding studies, however, reveals several methodological limitations. First, the reliability of the evaluation protocols is questionable. Contrary to the findings of Csibi et al. [2026], evaluation under the recommended inference settings shows that Qwen3-4B achieves higher scores than Racka-4B, its Hungarian-adapted version. Data contamination is evident in the work of Yang et al. [2025d] and Szentmihályi et al. [2025], potentially biasing the reported results. Second, the training pipelines fall short of current best practices in corpus curation and data mixture, which risks wasting substantial compute on low-quality data. The lack of controlled ablations prevents reliable assessment of these choices. Third, none of the three papers assessed forgetting or capability loss. Testing the adapted models on a subset of the original
benchmarks indicates performance decline in all three cases, especially Racka-4B. These observations emphasize the importance of rigorous experimental design in
language model development, given the significant computational and financial costs involved.
Read the full abstract ▾
Computation and Language
Too Good to Be Real? Diagnosing and Reducing the Gap Between AI Preference and Real User Engagement
Xinglang Zhang, Yuanmeng Xiang, Yunyao Zhang et al. (+3 more)
Large
language models are increasingly used to generate and evaluate online content, yet it remains unclear whether the qualities they associate with higher engagement match what real users respond to. We study this question using 1.17 million answers to 25,978 questions from Zhihu, Quora, and Reddit, comparing real platform answers and AI-generated answers across four within-question engagement levels. We introduce Ontological Preference Measurement, which represents answers along three dimensions: logic, affect, and expression. We find a systematic gap between AI preference and real user engagement: as target engagement increases, LLMs add more explicit logical structure, while real user engagement is more strongly associated with affective and expressive salience. We call this tendency logic overbinding. Based on this diagnosis, we propose Ontology-Masked
Reasoning Autoencoding (OMRA), a controlled intervention that masks and reconstructs over-explained spans while preserving stance, factual content, and coherence. Across four LLM families, OMRA reduces the measured gap by an average of 54.4%. In human evaluation, OMRA wins 62.4% of pairwise preference judgments against matched real platform answers, even though the real answers are more often judged to be human-written.
Read the full abstract ▾
All authors Xinglang Zhang, Yuanmeng Xiang, Yunyao Zhang, Zeliang Chen, Junqing Yu, Zikai Song
Computation and Language
I code or AI code: A comparative evaluation of AI-rated scores in classroom observations
Y. Fong, J. Xiang, T.Y.D. Chan et al. (+2 more)
Classroom observations are widely recognized as a key tool for establishing
benchmarks of education quality and guiding pedagogical improvement, yet they remain resource-intensive and dependent on trained observers. This study evaluated the feasibility of using a LLM (GPT-5 model) to score teacher-child interactions in early childhood classrooms, benchmarked against human raters. The study analyzed 87
video-recorded observations from 38 classrooms across 30 kindergartens in Hong Kong. Using observation transcripts, the AI model was configured to apply the full Classroom Assessment Scoring System (CLASS) framework. AI-rated scores were then compared with human ratings by examining correlations and differences in mean scores of the CLASS domains and dimensions. The results showed greater convergence between AI and raters for the Emotional Support domain and, in particular, the Quality of Feedback dimension, which captures how teachers use feedback to extend children's learning. Greater divergence emerged for interactions that were more procedural or context-dependent, particularly within the Classroom Organization and Instructional Support domains. These findings suggest that transcript-based AI scoring may capture some of the relative variation in teacher-child interactions but cannot yet reproduce calibrated human judgements consistently across the full CLASS framework. AI-assisted observation may therefore be more appropriate as a preliminary screening tool rather than as a replacement for trained observers, providing teachers with evidence for reflection rather than high-stakes evaluation. Future research should examine whether domain-specific training and incorporation of contextual and visual information can improve
alignment between AI and human rated scores.
Read the full abstract ▾
All authors Y. Fong, J. Xiang, T.Y.D. Chan, K. Lee, E.Y.H. Lau
Computation and Language · Cross-listed from cs.RO
M2Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models
Chunpu Xu, Zhixuan Liang, Yuhao Zhang et al. (+6 more)
Recent advancements have successfully adapted autoregressive
language models to process
multimodal signals, such as images and actions. Since raw action signals are continuous, effective tokenization is essential to map high-dimensional inputs into compact discrete
tokens for autoregressive processing. However, existing discrete action tokenizers often suffer from high reconstruction loss, failing to preserve the fine-grained dynamics required for precise control. This "discretization bottleneck" significantly limits the performance ceiling of downstream
Vision-
Language-Action (VLA) models. To address this, we propose ${M}^2$Tok, a Multi-head Multi-codebook Action Tokenizer designed to minimize reconstruction error and enhance policy performance. Our approach introduces two key structural innovations: (1) we decompose the latent action features into multiple heads, enabling the model to implicitly align specific heads with distinct action dimensions; (2) we assign independent codebooks to each head for quantization. By leveraging the combinatorial nature of multiple codebooks, we significantly expand the representational expressivity of the tokenizer, leading to substantially lower reconstruction loss compared to previous methods. We evaluate the ${M}^2$Tok-based VLA on the RoboTwin, Simpler-Env, and 3 zero-shot real-world tasks. Experimental results demonstrate our method not only achieves superior reconstruction fidelity but also significantly boosts the success rate of VLA models. Comprehensive ablation studies further confirm the effectiveness of the multi-head and multi-codebook mechanisms. Code is available at https://github.com/cpaaax/M2Tok.
Read the full abstract ▾
All authors Chunpu Xu, Zhixuan Liang, Yuhao Zhang, Chi-Min Chan, Jessie Wang, Yang Xiao, Mengkang Hu, Xiaokang Yang, Yao Mu