Computation and Language
MLLM-Microscope: Unlocking Hidden Structure Within Multimodal Large Language Models
Ravil Mussabayev and Rustam Mussabayev
This work presents MLLM-Microscope, a novel system designed for analyzing the hidden representations within
Multimodal Large
Language Models (MLLMs). Our system evaluates the linearity, intrinsic dimension, and anisotropy of
multimodal token embeddings across
transformer layers. Utilizing the ScienceQA
dataset, we evaluate two state-of-the-art MLLMs, LLaVA-NeXT and OmniFusion. We find that both the main and residual streams for
tokens of both modalities exhibit highly linear behaviors across
transformer layers. However, LLaVA-NeXT's image
tokens reveal a slight decline in linearity, whereas OmniFusion's remain consistent. Image
token dimensions in OmniFusion remain consistently higher across layers compared to LLaVA-NeXT. Also, the OmniFusion's anisotropy is observed to stay consistently low throughout the layers. These findings suggest that the inner workings of MLLMs highly depend on the nature of modality fusion performed before passing the
token sequence into LLM. This and other new potential insights obtainable from our system are surely capable of enhancing our understanding of the inner workings of MLLMs, informing future model design and optimization.
Read the full abstract ▾
Computation and Language
Citation Grounding Measures the Oracle: Graph Coverage Determines Reported LLM Hallucination Rates in Law
Volodymyr Ovcharov
Verifying LLM-generated legal citations against a graph of citations extracted from real court decisions is an appealing way to measure
hallucination at scale: no annotators, no reference answers. We show that what such a metric reports is governed by the coverage of the graph it queries rather than by the model it evaluates, and that at the coverage where its verdicts become trustworthy it stops distinguishing models at all. We score 400 responses (100 Ukrainian legal queries x four commercial LLMs) against two snapshots of the same national citation graph, holding responses, extractor and metric fixed. Against a sparse snapshot (4.7e5 records) citation grounding ranges 0.791-0.855, apparently showing 15-21% of citations hallucinated. Against a dense snapshot of the same registry re-derived ten weeks later (3.3e8 records, 5.8e7 decisions) the identical responses score 0.989-0.999. Subsampling shows the cause is coverage: harvesting modelled as uniform record sampling, calibrated on nothing but the record count, reproduces the sparse scores to within 0.018 while knowing nothing about the models. Bootstrapping over the queries shows the other half, which no version of this work reported: no pair of systems is separable at 95% at any oracle size tested. The metric is caught between two failures. Sparse oracles discriminate, but what they discriminate is harvesting coverage; dense oracles are trustworthy and separate nothing. Densifying the graph, the obvious remedy for the first, produces the second. An independent legislation registry adjudicates: all 54 citations flagged by the sparse oracle name real articles, a 100% false-positive rate; of the four flagged by the dense oracle, two are fabrications, one a coverage gap, and one we cannot classify - it looks like a repealed provision, which is the argument in miniature.
Read the full abstract ▾
Computation and Language
Chunking Methods on Retrieval-Augmented Generation - Effectiveness Evaluation Against Computational Cost and Limitations
Mateusz Śmigielski, Michał Rajkowski, Mateusz Zbrocki et al. (+5 more)
Retrieval-Augmented Generation (RAG) has demonstrated significant capabilities in enhancing the performance of Large
Language Models (LLMs). One of the key tasks in RAG systems is the chunking process. Traditionally, fixed-size chunking and semantic chunking have been the standard approaches. However, interest in chunking strategies has been increasing, leading to a growing number of proposed methods that often claim improved performance over these conventional techniques. Many of these approaches are tailored to specific use cases and data types, with limited evidence of their effectiveness across diverse scenarios. As a result, it remains challenging to directly compare different techniques and assess their relative strengths. To the best of our knowledge, this study is the first to systematically evaluate the effectiveness of a wide range of chunking methods and emphasize the underlying challenges of chunking strategies in RAG systems. While chunking is commonly treated as a simple preprocessing step, we show that it introduces a range of impactful and often overlooked issues.
Read the full abstract ▾
All authors Mateusz Śmigielski, Michał Rajkowski, Mateusz Zbrocki, Michał Bernacki-Janson, Karol Kunicki, Julianna Godziszewska, Maciej Piasecki, Konrad Wojtasik
Computation and Language
IDEAFix: Evaluation Framework for Creative Defixation Prompting in LLMs
F. Carichon, S. Sharma, M. Girard et al. (+2 more)
Large
language models (LLMs) are increasingly used for tasks involving creative problem solving and idea generation. However, there is a lack of consensus concerning their creative capabilities: some studies report superior performances compared to humans, while others highlight structural limitations such as fixation and the homogenization of outputs. Existing evaluation approaches either rely on narrow, decontextualized tasks that do not capture goal-oriented generation or on broader settings that confound multiple aspects of the creative process, making it difficult to isolate the effects of task formulation, prompting, and evaluation design. Significantly, the role of structured prompting strategies in shaping idea generation remains underexplored. Therefore, we introduce IDEAFix, an evaluation framework for analyzing divergent thinking in open-ended idea generation tasks. We
prompt models to generate multiple original solutions to controlled variations of short design scenarios, task attributes, and defixation prompting strategies. This design enables systematic analysis of how structured guidance influences LLMs' idea generation. Our results show that both task formulation and attribute selection significantly affect models' performance, and that simple prompting strategies can boost the originality of solutions. However, we also observe persistent output homogenization across models, confirming inherent limits in their ability to generate diverse solutions. Overall, IDEAFix provides a controlled, extensible framework for studying the mechanisms underlying LLMs' creativity.
Read the full abstract ▾
All authors F. Carichon, S. Sharma, M. Girard, R. Rampa, G. Farnadi
Computation and Language · Cross-listed from cs.SI
GenPT: Beyond Self-Report for Reliable LLM Psychometrics via Generative Projective Testing
Ming Wang, Shuang Wu, Bixuan Wang et al. (+7 more)
Self-report questionnaires remain the prevailing tool for probing the psychological states of persona-conditioned
agents (PC-
Agents). However, classical instruments inherit two well-known threats: contamination from training corpora and directional bias driven by social-desirability or contextual framing. To overcome these methodological bottlenecks, we ask whether projective paradigms can be adapted into a robust psychometric tool. We introduce GenPT (Generative Projective Testing), which reformulates TAT, Rorschach, and SCT with newly generated stimuli and organizes assessment as a three-stage pipeline to derive standardized psychological indicators and target states. Evaluating PC-
Agents induced via CharacterRAG and AnnaAgent profiles, we
benchmark GenPT's reliability and validity against classical questionnaires. The results indicate that questionnaires exhibit systematic directional shifts under social-desirability framing, most strongly on suicide ideation. In contrast, GenPT's collected behavioral patterns stay near the symmetric baseline. Furthermore, under a longitudinal counselling context, GenPT-based depression assessment shifts by roughly an order of magnitude more than the questionnaire counterpart when Qwen3 serves as the backbone. Overall, GenPT complements self-report methods in scenarios where contamination resistance, bias asymmetry, and context sensitivity matter. Code and stimuli can be found at https://github.com/sci-m-wang/GenPT.
Read the full abstract ▾
All authors Ming Wang, Shuang Wu, Bixuan Wang, Lu Lin, Yuxin Chen, Xiaocui Yang, Daling Wang, Shi Feng, Yifei Zhang, Yufan Sun
Computation and Language · Cross-listed from cs.SD
Sympatheia: Emotionally Adaptive Voice Assistant with Continuous Affect Conditioning
Sukru Samet Dindar, Riki Shimizu, Xilin Jiang et al. (+1 more)
Empathetic spoken dialogue systems must infer a user's emotional state to respond appropriately, yet everyday
speech often carries weak, neutral, or ambiguous affective cues. To address this, we introduce Sympatheia, a
speech-to-
speech dialogue framework conditioned on affect inferred from the user's
speech and, when available, explicit affect specifications provided as a continuous valence–arousal (VA) control signal by a
multimodal sensing module or user interface. To train our model, we construct Sympatheia-18k, an
emotion-conditioned synthetic spoken dialogue corpus with 12
emotion anchors. This
dataset includes an emotional split for learning affective
speech behavior, and a neutral split that pairs emotionally neutral queries with multiple
emotion-conditioned responses to isolate explicit
emotion control in emotionally ambiguous cases. Empirical results show that Sympatheia outperforms
speech conversational baselines in generating responses whose semantic content and spoken delivery are both emotionally appropriate. We further show that the same VA interface can integrate
emotion estimates from diverse sensing modules, including facial expression, biosignals, and textual affect descriptions, improving response
alignment when
speech alone provides limited emotional evidence. These results suggest that continuous affect conditioning is an effective practical step for building emotionally adaptive voice assistants.
Read the full abstract ▾
All authors Sukru Samet Dindar, Riki Shimizu, Xilin Jiang, Nima Mesgarani
Computation and Language
Momento: Evaluating Persistent Memory and Reasoning with Multi-Session Agentic Conversations
Adril Putra Merin, David Anugraha, Ayu Purwarianti et al. (+1 more)
Recent advances in agentic AI have enabled
agents to complete complex tasks through tool use,
reasoning, and multi-step planning. Yet existing
benchmarks evaluate
agents within a single session, ignoring past actions, stated preferences, and prior decisions that
agents must integrate to fulfill personalized user goals. We introduce Momento, a
benchmark for persistent agentic task completion in multi-session service environments, requiring
agents to take consequential, tool-mediated actions while resolving temporal dependencies and evolving user goals across sessions. Experimental results reveal that current
agents fail primarily through misestimation of user state, treating prior session history as a reliable proxy for current context rather than stale information requiring re-validation, highlighting a substantial gap between current
agent capabilities and realistic long-horizon human-
agent interaction.
Read the full abstract ▾
All authors Adril Putra Merin, David Anugraha, Ayu Purwarianti, Genta Indra Winata
Computation and Language
Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate
Xiqi Hao, Zengqing Wu, Yu-Xuan Qiu et al. (+4 more)
Multi-
agent debate (MAD) among large
language models (LLMs) is often judged by answer changes and convergence, yet these outcomes alone cannot establish that peers improved the answer. We study this gap with matched self-reflection, stance-only, and full-
reasoning conditions. With five GPT-4o
agents on MMLU-Pro disagreement questions, 37% of
agent-question pairs change even without peer information. Among pairs stable under this control, 29.3% adopt a peer answer after seeing peer stances. These adoptions include corrections, new errors, and changes between wrong answers. The outcomes vary with correct-answer availability. When a correct peer answer is available, correction exceeds harmful adoption after accounting for peer support in the primary setting. Full
reasoning improves individual accuracy but provides no additional majority-vote gain over self-reflection while using 57% more
tokens. Responses to constructed peer materials vary across GPT-4o, DeepSeek-V4-Flash, and Qwen3-Plus. An answer-agnostic intervention suppresses both harmful and beneficial peer adoption without a detectable accuracy gain. Together, the results show that convergence is an ambiguous signal of value and that debate must be judged by what peer information contributes to the final group answer.
Read the full abstract ▾
All authors Xiqi Hao, Zengqing Wu, Yu-Xuan Qiu, Chuan Xiao, Ruiqi Xu, Shuyuan Zheng, Jianbin Qin
Computation and Language · Cross-listed from cs.CR
Cross-Generational Transfer of Adversarial Attacks Reveals Non-Monotonic Safety Alignment in LLMs
Subhadip Mitra
Safety
alignment in LLMs does not improve monotonically across model generations. Studying four generations of Google's Gemma family (7B-31B) with quality-diversity
evolution (MAP-Elites) as an automated red-teaming probe, we find that Gemma 3 (12B) exhibits 68.7% +/- 5.7% attack success rate (ASR; mean +/- std, 3 seeds), significantly higher than its predecessor Gemma 2 (45.5% +/- 7.2%; p = 0.030, paired bootstrap) and its successor Gemma 4 (33.9% +/- 1.8%). Replaying evolved attack archives across generations reveals that attacks from other generations transfer to Gemma 3 at 44-46% but only 14-18% to Gemma 4, indicating that Gemma 4's safety gains generalize beyond the attack distributions evolved against earlier generations. Under our 8B judge, copyright and cybercrime vulnerabilities register at near-100% across all generations, though a second-judge audit (Section 6) suggests the copyright result is sensitive to judge choice. Misinformation ASR jumps from 29% to 99% between Gemma 2 and Gemma 3 and remains elevated at 77% in Gemma 4, indicating the regression was not fully addressed. These patterns are invisible to static
benchmarks and emerge only through adaptive, longitudinal probing. All experiments use 3 random seeds with a unified self-hosted judge; code and artifacts are available at https://github.com/bassrehab/red-queen.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.MA
Dynamic Coordination Strategy Selection for Enterprise Multi-Agent Systems
Thanh Luong Tuan
Enterprise multi-
agent systems increasingly expose multiple coordination patterns, but deployments often lack evidence for when to use consensus, debate, synthesis, or a simpler single-
agent workflow. This paper evaluates whether coordination strategy should be selected dynamically by problem class rather than fixed globally. We run a frozen
matrix of 30 enterprise tasks spanning six industries, five problem classes, four execution conditions, three replications per
cell, and four model arms: qwen_local, sonnet, gemma_openrouter, and an auxiliary openai cloud-validation arm. All 1,440 generated outputs are judged by a fixed Sonnet rubric. The main finding is bounded and operationally useful, but it is not the original strict H1. The pre-registered exact-winner/CI criterion is not supported: exact winner identity is unstable across model arms, and several predicted strategies are close to, but not above, the best observed alternative. A weaker near-best routing claim is strongly supported. In every pre-registered model arm and problem class, and again in the auxiliary OpenAI validation arm, the predicted strategy is within 0.10 quality-score points of the best observed condition. Structured compliance verification is the clearest exception to the original mapping: all arms favor single_
agent rather than consensus. A pre-registered Kendall's W test finds no reliable difference between Vietnamese-domain and English-domain tasks in how consistently the four coordination conditions are ranked (mean W of 0.20 in both strata; signed-rank p = .85), so H2 is not supported. We conclude that enterprise coordination policy should use dynamic routing as a calibrated default, not as a deterministic winner-selection law.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CR
Quality-Diversity Evolution for Discovering Diverse Vulnerabilities in LLM Safety
Subhadip Mitra
Current approaches to LLM
adversarial testing suffer from coverage gaps: manual red-teaming does not scale, LLM-as-attacker methods exhibit mode collapse, and gradient-based approaches produce uninterpretable gibberish. We introduce a quality-diversity evolutionary framework that operates at the semantic level, evolving interpretable attack strategies rather than
token sequences. Using MAP-Elites, we maintain a diverse archive of attacks across behavioral dimensions (strategy type, encoding method,
prompt length). In experiments across GPT-4o-mini, Claude 3.5 Sonnet, Gemini 2.0 Flash, and an open-weight coding model (Devstral-small-2), we discover distinct vulnerability profiles: GPT-4o-mini is vulnerable to hypothetical and multi-turn framing combined with ROT13 encoding (fitness 0.8), Gemini to direct attacks with ROT13 and multi-turn with Leetspeak (0.8), while Claude shows uniformly ambiguous responses across all strategies (max 0.4). The semantic representation produces interpretable attacks that reveal systematic, model-specific weaknesses, providing actionable insights for improving LLM safety and a reproducible baseline for evaluating future frontier models. Code and experiment artifacts are released at https://github.com/bassrehab/red-queen.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Confidence-Adaptive SwiGLU for Mixture-of-Experts
Shaohua Li, Xiuchao Sui, Xiaobing Sun et al. (+4 more)
SwiGLU has become a standard gated activation in modern
Transformer MLPs, yet its gate sharpness – the smoothness and selectivity of the gating function – is typically fixed throughout training. In this work, we propose Confidence-Aware SwiGLU ($\kappa$-SwiGLU), a variant of SwiGLU for Mixture-of-Experts (MoE) models that adjusts expert gate sharpness according to
token-level routing confidence. Specifically, $\kappa$-SwiGLU parameterizes the SiLU gate sharpness coefficient as a learnable function of the router logit, enabling each expert gate unit to interpolate between smooth, broadly active gating and sharp, selective gating. We evaluate $\kappa$-SwiGLU on the FineWeb-Edu
dataset across MoE
Transformer models ranging from 8 to 28 layers. Across these settings, $\kappa$-SwiGLU improves mean CORE performance while adding negligible parameters and incurring only a small computational overhead, demonstrating that confidence-aware gate sharpness is a promising mechanism for improving MoE MLPs. The code is available at https://github.com/askerlee/kappa-swiglu.
Read the full abstract ▾
All authors Shaohua Li, Xiuchao Sui, Xiaobing Sun, Yuhang Wu, Liangli Zhen, Yong Liu, Rick Siow Mong Goh
Computation and Language
Internalize the Temperature: On-Policy Self-Distillation as Policy Reheater for Reinforcement Learning
Xuewei Yang, Jiachen Yu, Jie Wu et al. (+3 more)
Reinforcement learning from verifiable rewards improves the
reasoning ability of large
language models, but often suffers from
entropy collapse, in which increasingly concentrated policies reduce rollout diversity and useful learning signals. Existing remedies either constrain the RL objective (e.g.,
entropy regularization) or adjust sampling temperature during rollout collection, but these interventions remain external to the model parameters. We propose Temperature-Scaled On-Policy Self-
Distillation (TS-OPSD), a lightweight policy reheating method that internalizes the exploratory effect of temperature into model parameters. Starting from an
entropy-collapsed RL checkpoint, TS-OPSD constructs a self-teacher by applying high-temperature scaling to the model's own logits, then distills the resulting smoother distribution back into the student. This policy reheating requires no external teacher, privileged data, or additional inference cost. Experiments on Qwen3-4B-Base and Qwen3-8B-Base show that policy reheating yields a stronger initialization for continued RL than both standard continued RL and rollout-level temperature reheating. Further analyses show that TS-OPSD mainly reduces output sharpness while preserving intermediate representations, top candidate sets, and
reasoning capability. These results suggest that
entropy restoration can serve as a simple post-collapse intervention for extending
reasoning-oriented RL.
Read the full abstract ▾
All authors Xuewei Yang, Jiachen Yu, Jie Wu, Shaoning Sun, Junjie Wang, Yujiu Yang
Computation and Language
I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applications
Dasen Dai, Biao Wu, Meng Fang et al. (+2 more)
Recent advances in visual
language models have enabled autonomous
agents for complex
reasoning, tool use, and document understanding. However, existing document
agents mainly transform papers into static artifacts such as summaries, webpages, or slides, which are insufficient for technical papers involving dynamic mechanisms and state transitions. In this work, we propose a Paper-to-Interactive-System
Agent that converts research papers into executable interactive web systems. Given a PDF paper, the
agent performs end-to-end processing without human intervention, including paper understanding, system modeling, and interactive webpage synthesis, enabling users to manipulate inputs and observe dynamic behaviors. To evaluate this task, we introduce a
benchmark of 19 research papers paired with expert-built interactive systems as ground truth. We further propose PaperVoyager, a structured generation framework that explicitly models mechanisms and interaction logic during synthesis. Experiments show that PaperVoyager significantly improves the quality of generated interactive systems, offering a new paradigm for interactive scientific paper understanding.
Read the full abstract ▾
All authors Dasen Dai, Biao Wu, Meng Fang, Shuoqi Li, Wenhao Wang
Computation and Language
From Empathy to Personalized Empathy: Adapting Empathetic Strategies to Individual Users
Wuqiang Zheng, Chengbing Wang, Yilin Yang et al. (+6 more)
As Large
Language Models (LLMs) are increasingly deployed in long-term interactions with users, empathy has become an increasingly important capability. However, existing research overlooks the influence of users' personality traits on empathetic strategies during long-term interactions. To address this gap, we introduce the task of personalized empathy, which focuses on adapting empathetic strategies according to users' personalized characteristics derived from history. To study and enhance this capability, we construct PersonaEmp, a personalized empathy
dataset built from long-term user-AI interactions, featuring rich user histories, persona information, and empathy-seeking queries. We further propose PereGRM, a reward modeling framework that combines the empathy evaluation structure with dynamic evaluation criteria generation for fine-grained reward modeling. Experimental results across different settings and multiple judge models show that PereGRM consistently achieves the strongest performance improvements, indicating its effectiveness for enhancing personalized empathetic capabilities.
Read the full abstract ▾
All authors Wuqiang Zheng, Chengbing Wang, Yilin Yang, Junyi Cheng, Jianfei Xiao, Hu Sun, Yi Xie, Yangyang Li, Wenjie Wang
Computation and Language
WaveFilter: Enhancing the Long-Context Capability of Diffusion LLMs via Wavelet-Guided KV Cache Filtering
Jinnan Yang, Yan Wang, Zhen Bi et al. (+5 more)
Diffusion Large
Language Models (DLMs) have demonstrated significant advantages across various tasks. However, constrained by their multi-step iterative inference mechanism, their computational overhead and inference latency in long-context tasks have become core bottlenecks restricting their large-scale deployment. When processing long sequences, existing Key-Value (KV) caching mechanisms often face a dilemma where generation quality degrades drastically, where the core challenge lies in precisely and efficiently filtering critical
tokens within ultra-long contexts. Inspired by the human reading process, we propose WaveFilter, a universal and training-free caching framework. This framework innovatively introduces the wavelet transform for decomposition of long sequences to achieve precise identification of key
tokens, based on which a sparse KV Cache is constructed to compute the final contextual representation. Experimental results demonstrate that WaveFilter, as a plug-and-play generic framework, significantly enhances the performance of existing mainstream KV Cache methods in complex long-context tasks.
Read the full abstract ▾
All authors Jinnan Yang, Yan Wang, Zhen Bi, Kehao Wu, Xiaojie Li, Jungang Lou, Zechao Li, Jing Liu
Computation and Language
EPIC: Efficient and Parallel Inference under CFG Constraints for Diffusion Language Models
Hyundong Jin and Yo-Sub Han
Controlling
language model outputs is essential for ensuring structural validity, reliability, and downstream usability, and
diffusion language models are no exception. Recent advances in
diffusion language model decoding have extended output control beyond regular constraints to context-free grammar (CFG) constraints. However, the prior CFG-constrained decoder can be up to four times slower than unconstrained decoding, forcing a trade-off between the correctness benefits of CFG constraints and the decoding efficiency of
diffusion models. A key source of this overhead is sequential validity checking, which limits parallel
token commitment and adds repeated validation costs. We propose EPIC, a CFG-constrained decoding framework designed to improve this efficiency-correctness trade-off. In order to reduce decoding overhead without sacrificing syntactic correctness, EPIC combines lexing memoization, relaxed compatible subset selection for parallel commit, and validation using Earley-style parsing instead of deterministic automata. This design enables multiple compatible
tokens to be committed together while avoiding repeated lexing and expensive validation. Experiments on three
benchmarks using four models show that EPIC improves the efficiency-correctness trade-off, bringing runtime close to unconstrained levels while maintaining comparable syntactic and functional correctness. Relative to the prior CFG-constrained decoder, EPIC achieves a best-case inference-time reduction of 67.2%. Our implementation is available at https://github.com/hyundong98/EPIC-Decoding .
Read the full abstract ▾
Computation and Language · Cross-listed from eess.AS
Local Diagnostics of Continuous Normalizing Flow for Out-of-Distribution Detection
Xinwei Cao, Mengxuan Lu, Torbjørn Svendsen et al. (+1 more)
We address the problem of out-of-distribution (OOD) detection for target observations embedded in a subspace of the high dimensional data space. Using continuous normalizing flows (CNFs), we propose a Lagrangian sub-flow (LSF) framework designed to isolate and estimate the density for the relevant components in the representation and using the remaining components as context. Through experimentation with models for
speech synthesis, we show that CNFs, similarly to other deep generative models (DGMs), are susceptible to the "likelihood paradox", where high likelihood is erroneously assigned to OOD samples. This is attributed to the inductive bias of DGMs that prioritize low-level structural details over high-level semantic coherence. To mitigate this phenomenon, we propose a number of
geometric diagnostic signals based on the velocity field over the sub-flow trajectory. Based on these signals, we design metrics for the challenging task of zero-shot phoneme-level mispronunciation detection. Finally, we demonstrate the superiority of these metrics compared to likelihood-based methods on a real-world mispronunciation detection
benchmark.
Read the full abstract ▾
All authors Xinwei Cao, Mengxuan Lu, Torbjørn Svendsen, Giampiero Salvi
Computation and Language
OCC-RAG: Optimal Cognitive Core for Faithful Question Answering
Maksim Savkin, Mikhail Goncharov, Alexander Gambashidze et al. (+7 more)
Recent progress in the development of
language models has been defined by scale, with each generation absorbing more of the world's knowledge into its weights. However, many practical applications benefit more from robust
reasoning than from extensive parametric knowledge. In this setting, task-specialized small
language models (SLMs) offer a principled design choice. We introduce Optimal Cognitive Core (OCC), a family of SLMs built around this premise. As a variant of OCC, we present OCC-RAG, optimized for faithful question answering (QA) grounded in the provided context. This task directly aligns with the OCC design approach, requiring multi-hop
reasoning over supplied passages while ignoring memorized knowledge. To train OCC-RAG, we implement a novel pipeline for synthesizing multi-context, multi-hop QA data at scale, producing a corpus of over three million examples targeting multi-hop
reasoning, strict context faithfulness, and calibrated abstention. We release OCC-RAG-0.6B and OCC-RAG-1.7B, both mid-trained on this corpus. The models produce structured
reasoning traces with source citations grounded in literal quotes from the context. Through OCC-RAG, we demonstrate that compact, task-specialized SLMs can match or exceed general-purpose models 2 – 6x their size across multi-hop
reasoning (HotpotQA, MuSiQue, TAT-QA), faithfulness (ConFiQA), and refusal (MuSiQue-Un)
benchmarks.
Read the full abstract ▾
All authors Maksim Savkin, Mikhail Goncharov, Alexander Gambashidze, Alla Chepurova, Dmitrii Tarasov, Nikita Andriianov, Daria Pugacheva, Vasily Konovalov, Andrey Galichin, Ivan Oseledets
Computation and Language · Cross-listed from cs.AI
Moxia: A Trust-First Neuro-Symbolic Execution Architecture for Self-Explaining Mathematical Reasoning
Alessio Bruno
We present Moxia (formerly AXIOM), a trust-first neuro-symbolic architecture for self-explaining mathematical
reasoning over natural-
language input. Its
language model is strictly a canonicalizer: it rewrites informal problem text into a narrow schema consumed by a deterministic Computer-
Algebra-System (CAS) pipeline, which derives and verifies the answer or abstains as a first-class output. Routing follows a 1:1:1
alignment of problem-shape regex, schema-specific
prompt, and closed-form CAS handler, with 4,783 routes shipped, 71% of which answer without invoking the
language model, and zero LOST_CORRECT regressions as a standing release gate. Because the answer is derived rather than generated, so is its explanation: every handler emits a step trace of the computation it performed, rendered as prose by a layer covering all 4,785 task files that cannot narrate a step the handler did not take. Derivations export to Lean 4 as well: 479 task files (10%) emit a
theorem from the problem's declared data, 445 accepted by the Lean kernel with Mathlib; that gate covers a fixture corpus, so live output is generated, not machine-checked. We report two numbers and never fuse them. On the full 7-category MATH test split, designed against, Moxia answers 90.2% (4,510/5,000) with one confident-wrong answer (99.98% trust on parseable). On held-out MATH-500, never designed against, it answers 89.2% (446/500) with zero confident-wrong answers. The 1.0 pp gap is the substantive result: a registry that had merely memorized problem shapes would collapse on held-out data, and this one does not. The rule-only path answers the 20,000-record lm-eval arithmetic
benchmark at 100%, 1 ms per record. What we emphasize is not an accuracy figure but the forward dynamic: every logged abstain is a candidate correct after one ship cycle, since new tasks compose without regressing the registry.
Read the full abstract ▾
Computation and Language
FineVerify: Scaling Test-Time Compute with Fine-Grained Self-Verification for Agentic Search
James Xu Zhao, Hui Chen, Bryan Hooi et al. (+1 more)
Agentic search requires
language model
agents to explore many sources and answer complex information-seeking questions. Scaling test-time compute is a promising way to improve these
agents, but current approaches can fail, because correct answers are often sparse and score-based selection depends on model calibration. We propose FineVerify, a fine-grained self-verification framework that decomposes each question into checkable sub-questions, verifies sampled candidates against each sub-question, and selects the candidate with the highest aggregated score. This per-check structure turns selection into simpler local judgments and produces scores under the same explicit criteria. Across four agentic search
benchmarks and two models, FineVerify consistently outperforms standard scaling baselines. With only four sampled trajectories, it improves GPT-5-mini by 8.2 accuracy points and Gemini-3-flash by 5.6% on average. With 12 samples, FineVerify enables GPT-5-mini to surpass frontier GPT-5 on BrowseComp-Plus. Beyond accuracy, FineVerify produces interpretable verification traces that help audit
benchmark errors, suggesting broader applications for inspecting agentic search systems. Code and data are available at https://github.com/XuZhao0/fineverify
Read the full abstract ▾
All authors James Xu Zhao, Hui Chen, Bryan Hooi, See-Kiong Ng
Computation and Language · Cross-listed from cs.LG
MESA: Improving MoE Safety Alignment via Decentralized Expertise
Yitong Sun, Yao Huang, Teng Li et al. (+5 more)
Mixture-of-Experts (MoE) architectures scale Large
Language Models (LLMs) efficiently, enabling greater capacity with reduced computational cost by dynamically routing inputs to relevant experts, yet introduce a critical vulnerability: Safety Sparsity, where safety capabilities concentrate in few experts, making them susceptible to
adversarial bypassing. Meanwhile, conventional
alignment methods uniformly adapt all parameters, ignoring their functional differences and inadvertently degrading performances. To address these challenges, we propose MESA (MoE Safety
Alignment), a targeted
alignment framework for MoE-based LLMs that strategically decentralizes safety responsibility to maximize coverage while minimizing interference with utility. Based on Optimal Transport (OT) theory, MESA operates through two mechanisms: (1) Expert Capacity Reallocation uses a transport cost
matrix to distribute safety duties to the most cost-effective experts, and (2) Dynamic Routing Refinement constrains the router to precisely activate these decentralized modules. Experiments show that MESA achieves robust defensive performance against varied harmful
benchmarks while preserving helpfulness. Code is available at https://github.com/lorraine021/MESA.
Read the full abstract ▾
All authors Yitong Sun, Yao Huang, Teng Li, Ranjie Duan, Yichi Zhang, Xingjun Ma, Hui Xue, Xingxing Wei
Computation and Language
LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification
Shefayat E Shams Adib, Ahmed Alfey Sani, Md Hasibur Rahman Alif et al. (+1 more)
Detecting psychological defense mechanisms in conversational text remains a challenging clinical NLP problem. For the PsyDefDetect 2026 shared task (nine-class utterance classification evaluated via macro F1), our team LinguIUTics achieves a macro F1-score of 0.3917 on the official positive-class leaderboard, ranking 4th out of 21 registered teams and improving over the Ministral-8B task baseline (31.48 macro F1) by 7.7 absolute points (24.4 percent relative). BERT-family encoders and zero-shot LLMs proved ineffective on rare classes due to severe class imbalance, leading us to QLoRA fine-tuning of Qwen3-8B. We leverage three key strategies: grouped stratified cross-validation (preventing leakage), minority-class round-robin lexical augmentation, and a post-processing pipeline with logit bias tuning and ensemble blending. Together, these components close much of the validation-to-leaderboard gap and substantially improve minority-class recall, driving the critical "Unclear" class (Level 8) from near-zero performance to an F1 score of 0.797.
Read the full abstract ▾
All authors Shefayat E Shams Adib, Ahmed Alfey Sani, Md Hasibur Rahman Alif, Ajwad Abrar
Computation and Language
French parsing enhanced with a word clustering method based on a syntactic lexicon
Anthony Sigogne, Matthieu Constant and Eric Laporte
This article evaluates the integration of data extracted from a French syntactic lexicon, the Lexicon-Grammar (Gross, 1994), into a probabilistic parser. We show that by applying clustering methods on verbs of the French Treebank (Abeillé et al., 2003), we obtain accurate performances on French with a parser based on a Probabilistic Context-Free Grammar (Petrov et al., 2006).
Read the full abstract ▾
Computation and Language
Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillation
Ruiqi Zhang, Lingxiang Wang and Hainan Zhang Zhiming Zheng
Self-
distillation improves learning efficiency by rewriting reference answers as training data that better matches the model's own distribution. However, reference answers also introduce strong stylistic biases, causing the generative model to imitate surface forms rather than learn useful
reasoning patterns. We observe that the rewriting data contains a large number of high-perplexity (PPL)
tokens, coming from two distinct sources: beneficial knowledge-enhancing logical corrections, and harmful stylistic drift induced by reference imitation. Treating all such
tokens equally can disrupt the base model's original distribution and degrade performance, especially on difficult
reasoning tasks. To address this, we propose Distribution-Aligned Self-
Distillation (DASD), which uses an answer-aware reference model to generate candidate
tokens and dynamically filters them according to the base model's confidence. DASD preserves
tokens that encode useful logical knowledge while suppressing distributionally misaligned style noise. Experiments on math, code, and commonsense
reasoning benchmarks show that DASD consistently outperforms competitive baselines, reduces high-PPL
tokens, and improves robustness across tasks of varying difficulty.
Read the full abstract ▾
Computation and Language
MemPro: Agentic Memory Systems as Evolvable Programs
Qingshan Liu, Guoqing Wang, Wen Wu et al. (+5 more)
Long-horizon autonomous
agents require
memory systems to retain historical information, track evolving states, and reuse relevant knowledge beyond finite context windows. Existing agentic
memory systems typically follow a
memory construction-
retrieval (MCR) pipeline, but often adapt mainly the
memory bank while keeping the surrounding pipeline fixed after deployment. This fixed-pipeline design struggles to handle heterogeneous task-specific failure modes and can become misaligned with
memory banks that evolve in scale and structure over time. To address these limitations, we propose MemPro, a system-level
evolution framework that treats the entire MCR pipeline as an evolvable program rather than adapting only the
memory bank or
prompt text. MemPro maintains a version tree of runnable
memory-system implementations, where an Evolving
Agent iteratively selects promising versions, diagnoses recurring failures, and creates improved child versions through failure-mode-guided edit-debug refinement. Experiments on LongMemEval, LoCoMo, HotpotQA, and NarrativeQA show that MemPro consistently outperforms strong static and
prompt-level evolving baselines within a few iterations, continues to improve with
evolution, and achieves a favorable performance-cost trade-off. Code is available at https://github.com/wanghai673/MemPro.
Read the full abstract ▾
All authors Qingshan Liu, Guoqing Wang, Wen Wu, Jingqi Huang, Xinqi Tao, Dejia Song, Jie Zhou, Liang He
Computation and Language
Linguistics-Aware Non-Distortionary LLM Watermarking
Shinwoo Park, Hyejin Park, Hyeseon An et al. (+1 more)
Watermarking should identify
language-model output without degrading quality or limiting verification to the model provider.
Multilingual deployment makes this harder because morphology,
segmentation, and script change where watermark evidence can be naturally embedded. We introduce LUNA, a linguistically adaptive watermark that combines model-free detection with single-
token non-distortion under the standard random-key model. LUNA estimates normalized next-tag
entropy from part-of-
speech contexts in an external corpus and uses it to set the depth of a non-distortionary binary tournament sampler; the detector reconstructs the same schedule from text, a tokenizer, a tagger, and a secret key. We evaluate LUNA on six typologically diverse
languages and two domains against eight primary baselines. LUNA attains an AUROC of 0.9959 and the lowest mean absolute median perplexity shift, 0.045, across the 12 settings; its 95% bootstrap interval [0.022, 0.073] lies below all baseline intervals. LUNA also records the lowest mean on Self-BLEU, Distinct-1, surprisal, and
entropy shifts; it is the only method that simultaneously achieves AUROC > 0.99 and an absolute median perplexity shift below 0.1 in a majority of settings, reaching this regime in 9 of the 12 settings while no baseline reaches it in more than 2. Our code is available at https://github.com/Shinwoo-Park/luna_watermark.
Read the full abstract ▾
All authors Shinwoo Park, Hyejin Park, Hyeseon An, Yo-Sub Han
Computation and Language
Toward Responsible and Epistemically Grounded Multilingual LLMs for Computational Social Science and Humanities
Wajdi Zaghouani
Large
language models have rapidly evolved in
multilingual competence and
reasoning capacity, enabling their integration into Social Sciences and Humanities research workflows. Yet existing evaluation paradigms remain anchored in task-based NLP
benchmarks and fail to address interpretive validity, cultural situatedness, and epistemic mediation. This paper reconceptualizes
multilingual reasoning LLMs as hermeneutic instruments that actively structure meaning production across linguistic and cultural contexts. Drawing on hermeneutics, philosophy of technology, science and technology studies,
multilingual NLP research, and computational social science methodology, we develop a theoretically grounded framework for evaluating
multilingual reasoning in Social Sciences and Humanities (SSH) research. We articulate a rigorous experimental protocol with operationalized metrics for cultural
alignment, cross-lingual stability, and
reasoning faithfulness, along with transparency requirements tailored to interpretive research tasks. We illustrate the framework through a concrete application scenario involving
multilingual political discourse analysis. The paper contributes a conceptual and methodological foundation for responsible integration of
multilingual reasoning LLMs into computational social science infrastructures.
Read the full abstract ▾
Computation and Language
SPADER: Step-wise Peer Advantage with Diversity-Aware Exploration Rewards for Multi-Answer Question Answering
Qiming Shi, Zhaolu Kang, Yunfan Zhou et al. (+2 more)
Large
language models are increasingly deployed as tool-augmented
agents to acquire information beyond parametric knowledge. While recent work has improved long-horizon tool-use
reasoning, most approaches focus on tasks with a single correct answer. In contrast, many real-world queries require discovering a comprehensive set of valid answers, a setting known as Multi-Answer QA. This setting raises two challenges: fine-grained credit assignment over long search trajectories and reward
alignment for sustained exploration beyond easy high-frequency entities. We propose SPADER, a reinforcement learning framework for long-horizon tool use in Multi-Answer QA. SPADER includes Step-wise Peer Advantage (SPA), a critic-free step-level credit assignment mechanism that aligns parallel trajectories by decision step and estimates advantages from peer returns. It also includes a diversity-aware exploration reward that promotes long-tail entity discovery by upweighting rare findings and downweighting redundant ones. Experiments on QAMPARI, Mintaka, WebQSP, and QUEST show that SPADER generally improves recall and overall F1 over prompting-based
agents, outcome-supervised RL methods, and recent step-level supervision approaches. Our code and model weights are available at https://github.com/KhanCold/spader.
Read the full abstract ▾
All authors Qiming Shi, Zhaolu Kang, Yunfan Zhou, Di Weng, Yingcai Wu
Computation and Language
Sandboxed Coding Agents are Competitive Omni-modal Task Solvers
Dongping Chen, Xuanao Huang, Zhihan Hu et al. (+3 more)
As
multimodal LLMs increasingly target
video and audio, it is often assumed that such tasks require native omnimodal models. We show that this is not always the case: coding
agents with only text+image access and a sandboxed tool-use interface can match, and in several settings outperform, SOTA native omnimodal models and predefined
multimodal agent scaffolds across multiple audio-
video benchmarks. Our trajectory analysis suggests that their strength comes from writing code and orchestrating tools to extract relevant evidence from transcripts, frames, and other modality signals, thereby converting omnimodal tasks into
retrieval and information-processing problems rather than ingesting entire media streams. We further characterize their limitations through a failure taxonomy and process-level trace analysis, and show that simple skill injection, including human-written and self-distilled skills, substantially improves performance. To explore open-source elicitation, we introduce Code-X, a training recipe with the OmniCoding trajectory
dataset and verifiable reward, and provide baselines on Qwen-3.5-9B and Qwen-3.6-27B. Finally, we argue that the next frontier is many-modality processing, and introduce TerminalBench-O, a process-level
benchmark for real-world omnimodal processing tasks. Code will be available at https://github.com/Dongping-Chen/OmniCoding.
Read the full abstract ▾
All authors Dongping Chen, Xuanao Huang, Zhihan Hu, Qingyuan Shi, Dianqi Li, Tianyi Zhou