Computation and Language · Cross-listed from cs.AI
Smart Content Ingestion for Generative AI Workloads
Abbas Raza Ali, Muhammad Ajmal Siddiqui and Moona Zahid
The
evolution of machine learning has progressively changed where intelligence resides in an AI system. In conventional machine learning the task, data representation, labels and model architecture were tightly coupled, so data preparation was narrow, schema-bound and visible. Generative AI decouples the model from any single task: one foundation model serves open-ended downstream tasks, and the generality gained on the model side is matched by heterogeneity on the data side, because enterprise knowledge is authored in the formats people use (PDF, presentations, spreadsheets, scanned documents, forms, tables, diagrams and mixed-layout files) that carry textual, visual,
geometric and structural information at once. A
language model or retriever cannot reason reliably over information misrepresented at this interface, so content extraction becomes a lifecycle stage in its own right whose errors no downstream retriever or re-ranker can repair. This paper presents a production-ready content-extraction system that makes this stage explicit, configurable, and measurable. The system incorporates selective OCR routing, a scarcity-first curation engine with a reference-based extraction scorer that measures character, word, and table-structure accuracy, a deterministic structure-aware parent-child chunker, and a read-only
retrieval evaluator that generates grounded questions from every page and reports Hit@k, mean reciprocal rank, and latency. On a 180-document corpus the best extractor scores 97.4 of 100 (character error rate 0.13%, table similarity 0.995) and the chunker reaches hit@1 of 68.6%, hit@10 of 92.8% and MRR 0.77 over 25,050 generated questions. We distil three design principles (structure before semantics, never mutate what you measure, budget your labels) and position measured content extraction as the perception layer of enterprise agentic systems.
Read the full abstract ▾
Computation and Language
Turnslide: Scalable Multi-Turn Data Synthesis by Walking a Finite-State Machine
Aaron Fainman, Gabriela Kadlecová, Maciej Gryka et al. (+7 more)
Small
language models are inexpensive to serve and can run on private infrastructure, but base models are often not good enough at multi-turn tool calling, and fine-tuning them needs per-API data that rarely exists. Existing synthesis methods are too expensive for high-scale fine-tuning, as they often require mock operational environments for different domains and multiple LLM calls per generated conversation turn. We introduce a fully automated, lightweight synthesis framework that models each API as a finite-state machine, representing the system as abstract states that determine when each tool may be called, producing state-valid sequences of tools; sequences are translated into complete examples with a single LLM call. Rather than optimize diversity, we set a target distribution over the number of turns, the tool sequence and task complexity. We measure data quality by fine-tuning SLMs on generated trajectories, showing that our FSM-based generation significantly improves downstream accuracy over an unmutated baseline and, against existing works, reaches 70.7% full accuracy over 63.4% and 53.7% with 3.6-6.6$\times$ fewer
tokens.
Read the full abstract ▾
All authors Aaron Fainman, Gabriela Kadlecová, Maciej Gryka, Bartosz Kruszczyński, Usman Zafar, Cédric Archambeau, Aaron Klein, David Salinas, Selim Nowicki, Jacek Golebiowski
Computation and Language · Cross-listed from cs.LG
Learning to Simulate Individuals from Macro Social Signals
Yining Zhao, Bushi Liu, Haofei Yu et al. (+5 more)
Large
language models are increasingly used to simulate how individuals respond to new situations, yet the behavioral
reasoning behind these responses is either inherited from pretraining or learned from individual-level annotations, which offer limited behavioral diversity and little supervision of the
reasoning itself. We propose to learn behavioral
reasoning from prediction
markets, whose price trajectories record how populations respond to real-world events at scale. We introduce macro2mind, which trains a
language model with GRPO using
market signals. A social behavioral decomposition makes behavioral
reasoning an explicit step of forecasting: the model infers representative groups of
market participants, predicts how each interprets the news and updates its beliefs, reasons about their interactions, and aggregates these responses into a price. A hindsight-regret curriculum with difficulty-aware sampling focuses training on transitions where hindsight-identified groups substantially improve the forecast while prioritizing examples that remain learnable for the current policy. The learned
reasoning applies to user simulation without further training. On SWM-Bench, macro2mind achieves state-of-the-art directional accuracy and correlation on Polymarket. Trained on
market data, it transfers zero-shot to four user-simulation
benchmarks (Humanual, OvertonBench, PRISM, and CAD) and has competitive performance among zero-shot methods. Used as a data generator, macro2mind also raises a downstream simulator's accuracy on unseen users by 15.5 points, outperforming data generated by its backbone by 13.2 points.
Read the full abstract ▾
All authors Yining Zhao, Bushi Liu, Haofei Yu, Zhengyang Qi, Shanyong Wang, Chuyue Li, Yuxiang Liu, Jiaxuan You
Computation and Language · Cross-listed from eess.AS
SEAL: Mixture-Closed Additive Reconstruction and Refinement-Aware Expert Routing for Efficient Speech Separation
Shao-Chun Hu, Zi-Xiang Lin, Jeih-Weih Hung et al. (+1 more)
Compact time-frequency separators that mask the mixture and refine through a shared
cell face two limits. First, a bounded multiplicative mask only scales a mixture bin, so where overlapping components cancel, the estimate stays small. Second, a shared
cell applies the same weights to every time-frequency
token at every step, so enlarging it adds compute everywhere. We present SEAL (Sparse Expert routing with Additive Latent reconstruction) to address both. For reconstruction, a zero-sum additive residual bounded by the local mixture amplitude lets estimates be nonzero where components cancel yet still sum to the mixture. For routing, a query built from acoustic and inter-step evidence sends each
token to one of six residual experts, and a norm cap keeps the step cue from overriding clear acoustic evidence. On EchoSet, SEAL (small) surpasses TIGER (small) by 0.31 dB SI-SDRi with 28% fewer parameters and 2.9 times fewer MACs, and SEAL (large) is within 0.07 dB SI-SDRi of TIGER (large) at 3.1 times fewer MACs.
Read the full abstract ▾
All authors Shao-Chun Hu, Zi-Xiang Lin, Jeih-Weih Hung, Hung-Shin Lee
Computation and Language · Cross-listed from eess.AS
GIVE-KWS: Gated Injection of Visual Evidence for Noise-Robust Query-by-Example Keyword Spotting
Ming-Hsiang Hu, Kuan-Tang Huang, Hung-Shin Lee et al. (+1 more)
Visual
speech promises noise-robust keyword spotting, yet a visual stream is not necessarily used. On a tri-modal query-by-example keyword spotting (QbyE-KWS)
benchmark, we find that a system with a task-trained visual encoder comes within 2 percentage points of a text-and-audio system in equal error rate (EER) at -10 dB, and link this gap to the encoder's lack of phonemic information. We present GIVE-KWS, whose fusion stage, GIVE (Gated Injection of Visual Evidence), conditions query audio on lip motion through gated cross-
attention. We show that visual robustness depends on two interacting conditions: a phoneme-bearing visual representation, and fusion that injects visual evidence rather than rescaling audio features. Under a phoneme-bearing encoder, injection yields an effective SNR gain of 4.0-9.3 dB over masking at -10 dB, whereas under a phoneme-poor one it nearly vanishes. Relative to the
benchmark system, GIVE-KWS reduces unseen-keyword EER by 72.9% at -10 dB and 62.8% on average.
Read the full abstract ▾
All authors Ming-Hsiang Hu, Kuan-Tang Huang, Hung-Shin Lee, Berlin Chen
Computation and Language · Cross-listed from cs.LG
Base Models Can Reason By Taking a Cue From Training Data
Sophie L. Wang, Amil Dravid, Rulin Shao et al. (+3 more)
In this paper, we study how training data creates associations between the
tokens at the start of a base model's response and the
reasoning behavior that follows. First, we demonstrate that fixing particular starting
token cues makes a base model's performance competitive with that of its reinforcement learning (RL)-trained counterparts on math and coding. For instance, the cue ".\n\nOkay" raises Olmo-3-7B's MATH-500 pass@1 accuracy from 42% to 78%, while "Alright," raises Qwen3-14B's from 72% to 87%. Second, RL makes these cues more likely, while fixing them recovers much of its performance gain over the base model. Third, we trace the
reasoning effects of
token cues to the training data. We perform causal data interventions to turn an arbitrary word, such as "chicken", into an effective
reasoning cue, or remove an existing cue's effect. A similar edit makes the
prompt instruction "Think duck duck goose" as effective as "Think step by step" at eliciting
reasoning. We also find that the hidden state representations induced by different cues correlate with different document types from the training set. Finally, we extend our study of
token cues with a case study in
language model safety, finding that different cues elicit distinct refusal and compliance behaviors that correspond to different types of training data.
Read the full abstract ▾
All authors Sophie L. Wang, Amil Dravid, Rulin Shao, Kevin Farhat, Sewon Min, Alexei A. Efros
Computation and Language · Cross-listed from cs.RO
Recursive Video In-Context Learning for Agentic Robot
Wenrui Bao, Xinxin Liu, Bingxin Xu et al. (+1 more)
LLM
agents that orchestrate frozen
vision-
language-action (VLA) policies improve across episodes through text
memory, which records what the
agent did but not how the task is done. A demonstration
video shows it, but fits poorly into an
agent's context. The full
video slows every turn, fixed keyframes lose the contact detail that decides whether a grasp holds, and what the
agent needs shifts from the task's structure while planning to the frames around each contact. We introduce Recursive
Video In-Context Learning (RV-ICL), a training-free method that turns a demonstration into a hierarchy the
agent navigates rather than a
prompt it receives. The hierarchy is built from the sub-events of the demonstration, such as grasps and releases. Its levels grow finer, from keyframes of the whole task to phases, moments and short clips, and are exposed through read-only tools. The
agent reads the coarse levels before planning. During execution it re-enters the hierarchy whenever a step needs more detail and loads only the clip of its current sub-goal. One demonstration per task is enough. Built on RPent, RV-ICL raises success from 92.6% to 96.5% on LIBERO-PRO and from 86.7% to 95.8% on LIBERO-Plus.
Read the full abstract ▾
All authors Wenrui Bao, Xinxin Liu, Bingxin Xu, Yuzhang Shang
Computation and Language
MemPilot: Orchestrating On-Demand Multimodal Memory Curation for LLM Agents
Haozhen Zhang, Haodong Yue, Quanyu Long et al. (+6 more)
Memory has become integral to the LLM
agent ecosystem, supporting information retention and reuse across interactions. However, most existing
agent memory systems construct
memory in a query-agnostic manner, which can incur unnecessary preprocessing cost and discard details that later prove essential. Recent studies have begun shifting
memory processing toward runtime adaptation, but typically specialize in particular operations or fixed processing schemes, leaving flexible control over performance, cost, and latency largely underexplored. To address this challenge, we present MemPilot, a flexible framework that orchestrates on-demand
memory curation under different performance–cost–latency preferences. Specifically, we optimize a multi-step LLM policy via reinforcement learning to iteratively choose between retrieving from query-agnostic
memory and delegating query-specific curation of raw
multimodal history to heterogeneous LLMs and VLMs. The policy jointly controls evidence amount, curation instructions, model selection, and visual access, enabling fine-grained allocation of runtime computation. To optimize this policy under competing objectives, we adapt objective-wise advantage decoupling by separately estimating each objective's advantage before aggregation. Moreover, we introduce prefix-based marginal utility estimation for fine-grained credit assignment across multi-step rollouts. Experiments on five
multimodal agent-
memory benchmarks demonstrate favorable performance–cost–latency trade-offs across optimization preferences, with preference sweeps yielding broader frontiers than existing trade-off-aware baselines.
Read the full abstract ▾
All authors Haozhen Zhang, Haodong Yue, Quanyu Long, Jianzhu Bao, Qingyuan Liu, Tao Feng, Bohan Liu, Weida Liang, Wenya Wang
Computation and Language
CLIFT: Conformal Self-Verification for Web Agent Training and Test-Time Scaling
Yifan Zhang, Yutong Dai, Viraj Prabhu et al. (+3 more)
Open-source web
agents are now strong enough to execute realistic browser tasks, but training them with reinforcement learning still depends on weak supervision: binary task success is too sparse for credit assignment, while frontier-
language-model judges are too expensive to call at every step and cannot be assumed available at deployment. We introduce CLIFT, a training and test-time scaling method built around conformal self-verification. During training, the
agent answers natural-
language verification questions about its own rollouts; a Compositional Conformal Certifier keeps only question signals whose URL-conditional evidence agrees with a training-time judge, assigns signed trust weights through polarity-aware lift, and blends the resulting verifier score into per-step rewards in a way that never subtracts from the judge baseline. At test time, the same certified bank is frozen and reused as structured evidence for Conformal Trajectory Selection (CTS): the
agent samples a greedy rollout and one or more diverse retries, the self-verifier summarises each URL trace, and a conservative majority-vote rule chooses whether to swap away from the current incumbent without calling any external judge. This single mechanism supports three settings. On WebArena Infinity, CLIFT achieves state-of-the-art performance among open-source web
agents. On VisualWebArena, a bank trained with the open model transfers to GPT-5.5 at test time and reaches state-of-the-art performance under the canonical harness. On Online Mind2Web, without training an
agent on the
benchmark, translating the certified question bank improves a live-web
agent in zero-shot evaluation. Together these results position conformal self-verification as a way to turn costly judge feedback into a reusable training signal and a judge-free test-time scaling signal.
Read the full abstract ▾
All authors Yifan Zhang, Yutong Dai, Viraj Prabhu, Zhiyuan Hu, Ran Xu, Zeyuan Chen
Computation and Language
PlotGround: Grounding Plot Digitization in Real Scientific Figures and Their Source Data
Yaohui Zhang, Binxu Li, Haoyi Duan et al. (+7 more)
Scientific figures often encode quantitative results that are not readily available in machine-readable form, making accurate plot digitization important for verifying and reusing published findings. Yet it remains unclear how accurately current models recover plotted values from real scientific figures, as existing
benchmarks rely largely on synthetic charts or cover only a limited range of chart types. We introduce PlotGround, an automated pipeline for building plot digitization
benchmarks from real scientific figures and their author-released source data. PlotGround maps figures to source tables, identifies reconstructable panels, and generates quantitative questions with source-grounded reference values. We use PlotGround to construct PlotGround-1k, a human-verified
benchmark of 1,119 questions from 1,066 bioRxiv preprints. Across sixteen
multimodal models, the best reaches 87.5% accuracy at a $\pm 5\%$ relative-error tolerance. Tightening the tolerance to $\pm 2\%$ lowers every model's accuracy by 11-24 percentage points, revealing a gap between approximate visual reading and precise quantitative recovery. PlotGround's paired figure-source structure lets us compare how accurately the same values are recovered from figures and from source tables. Providing source tables instead of figures raises a coding
agent's accuracy from 90.0% to 97.4% while cutting cost by 72%.
Read the full abstract ▾
All authors Yaohui Zhang, Binxu Li, Haoyi Duan, Jiacheng Miao, Yixin Wang, Xinran Du, Chenyue Li, Shilong Liu, Kevin Wu, James Zou
Computation and Language · Cross-listed from cs.SD
Paradee: Distilling Kokoro-82M into an 8M-Parameter Single-Voice Text-to-Speech Model
Sahil Mahendrakar
We distill Kokoro-82M, a widely used open text-to-
speech model with 54 voices, into Paradee, an 8.07M-parameter model that speaks one of them. Paradee keeps Kokoro's architecture with much narrower layers, and each of its two halves is trained separately against the frozen teacher. It has 10x fewer parameters and needs 15x less compute. We first synthesize a corpus with the teacher and keep its durations, pitch, energy and phoneme features. We then train a small text side to predict these values, and a small decoder to turn the teacher's saved values into the teacher's audio, first with spectral losses and then adversarially. Finally, we connect the two halves and quantize the weights to int8. It needs no
alignment learning and no joint training, and it runs on one laptop. Stored in int8, Paradee is 8.5 MB, runs 25x faster than real time on one CPU thread, and scores 4.41 on UTMOS against the teacher's 4.52. The student initially kept a slight buzz, which we trace to the phase of voiced
speech between 2 and 8 kHz. A phase-locking filter applied after synthesis removes most of it, with no training and no extra parameters. Code, model files and audio samples are at https://github.com/sahilmahendrakar/paradee
Read the full abstract ▾
Computation and Language
T-Search: An Open Agentic Retriever and Playground for Hard Multi-Step Search
Olga Tsymboi, Ramil Latypov, Aleksandr Medvedev et al. (+5 more)
We present T-Search, an open-weight agentic retriever for hard multi-step search. Given a question and a search tool over a fixed corpus, it runs a bounded multi-round search and returns a ranked list of evidence chunks with short justifications, leaving answer generation to a downstream model, so backend and generator can be swapped without retraining. T-Search is built on Qwen3.6-35B-A3B and trained on adversarially filtered synthetic search tasks with round-sliced supervised fine-tuning followed by GSPO on a recall reward. Averaged over seven English and Russian
benchmarks with gold evidence annotations, it reaches 56.0 Recall@10 with one rollout, 14.4 points above its base, and 61.3 with three fused rollouts, outperforming larger open models. We release the model, harness, live demo, and three
benchmarks, including TRuST, the first native-Russian hard-search
benchmark.
Read the full abstract ▾
All authors Olga Tsymboi, Ramil Latypov, Aleksandr Medvedev, Danil Taranets, Dmitrii Stoianov, Nikita Gulyakov, Gleb Alektorov, Anatolii Potapov
Computation and Language
IdeaLens: Detecting AI Ideas in Long-form Writing
Rishanth Rajendhran, Minjoon Choi, Jenna Russell et al. (+5 more)
While modern AI detectors identify who wrote the words, emerging policies on AI use increasingly hinge on a different question: who came up with the ideas? We introduce IdeaLens, a detector that identifies whether a document's ideas came from a human or AI (idea provenance), regardless of who wrote its words. To focus IdeaLens on ideas rather than prose, we represent documents as outlines: lists of items that each pair a discourse role with a brief, paraphrased description of the content, minimizing word-level overlap with the raw text. We train IdeaLens on 1M FineWeb documents with silver labels from Pangram, a prose provenance detector. Since the outlines are largely stripped of surface-level information, the labels must be fit mainly through the ideas. In a controlled study, IdeaLens's AI flag rate drops from 95% to 7% as models write from increasingly detailed human plans, while Pangram 4 still flags 92%; from AI-derived plans, IdeaLens stays above 96%. Conversely, on a new
dataset of 50 stories that human authors wrote from AI-generated plans, IdeaLens flags 68% of the stories as AI, compared to 8% for Pangram 4. On a comprehensive suite of 19 existing detection
benchmarks, we show that IdeaLens maintains strong detection rates at low false positive rates, suggesting that ideas themselves provide a powerful discriminative signal, and its performance holds across domains, formats, and
languages. Finally, we examine 90K predictions from IdeaLens to characterize systematic differences between human and AI ideation. We release our models and labeled
datasets to facilitate future research on idea provenance detection.
Read the full abstract ▾
All authors Rishanth Rajendhran, Minjoon Choi, Jenna Russell, Ramya Namuduri, Deniz Bölöni-Turgut, Marzena Karpinska, John Wieting, Mohit Iyyer
Computation and Language
Balancing Memory Pathways: Analyzing and Improving Memory Utilization in Hybrid LMs
Hyunji Lee, Joykirat Singh, Zaid Khan et al. (+5 more)
Recurrent-
attention hybrid
language models (LMs), which interleave
attention and recurrent layers, are increasingly used to combine the efficiency of the recurrent layers with the strong performance of
attention layers. Prior work suggests that
attention and recurrent layers offer complementary pathways to use past information:
attention supports precise
memory recall from earlier
tokens, while recurrent layers support consolidation of disparate information over long contexts. However, we observe that simply having access to both pathways does not mean that hybrid LMs are effectively using them. We find that they rely substantially more on
attention than on the recurrent state. Standard supervised fine-tuning improves overall performance but does not improve how the two
memory pathways are coordinated: the model becomes more reliant on information propagated by
attention layers, while its use of information propagated by recurrent layers remains limited. To encourage better coordination between the two
memory pathways, we add an auxiliary loss that limits
attention's access to earlier context while the recurrent state propagates through the full sequence. This objective encourages the model to retain and use information through the recurrent pathway alongside
attention. It improves overall performance, with particularly strong gains on tasks involving longer contexts or requiring information aggregation, consistent with the strengths of recurrent layers observed in analysis. Crucially, this imbalance and the benefit of our auxiliary loss generalize: they apply to multiple recurrent-
attention LMs in question-answering and agentic tasks, as well as to
attention-based LMs that combine different forms of
memory. Together, our findings show that simply providing multiple
memory pathways does not ensure their effective use, and that targeted supervision is needed to better coordinate them.
Read the full abstract ▾
All authors Hyunji Lee, Joykirat Singh, Zaid Khan, Justin Chih-Yao Chen, Elias Stengel-Eskin, Alessandro Sordoni, Arman Cohan, Mohit Bansal
Computation and Language
ufakzeka-karar: An Open Turkish Typed-Decision Model with Order-Invariant Option Scoring
Sait Furkan Teke
ufakzeka-karar is an open Turkish decision model with 182,494,466 parameters. Given a Turkish text and questions of a fixed answer type (a choice, a level on an ordered scale, or yes or no), it returns a temperature-scaled probability for every option and an expected error that serves as a "not sure" signal, without generating text and in one CPU forward pass for up to ten options. Built on the lab's ufakzeka-1-base, its head scores each option blind to the others at shared positions, so the answer does not depend on option order. A sequential head trained with shuffled options was about as accurate but changed 2.3 to 2.8 percent of its answers when only the option order changed; REINFORCE lost 10.2 points (0.102) of macro F1 to cross-
entropy. On the open set of HakemBench v1.0 (4,275 questions, 7 tracks) the released model ranks 7th of 16 rows with a composite of 0.660 (95% interval 0.642 to 0.677). Temperature scaling lowers calibration error (smooth ECE) on the development set but raises it on held-out support questions, from 0.027 to 0.045 for the first scored run, which never trained on them; the released model later trained on them, so its 0.036 to 0.064 is not an unseen-question test. The released model is the last of three runs scored on HakemBench, and its numbers are not blind. The second run's new training data was aimed at the first run's errors on the full test set in guardrails, moderation and customer support, and the released run was trained after the second run's guardrail results on the full test set were read, under a protocol fixed in writing before any of its data, code or runs. All its numbers come after these readings; its guardrail, moderation and customer support numbers carry the flag "shaped by reading the test results". With every model scored on the other four tracks only, its composite is 0.678, 6th of 16. Weights and code are under Apache-2.0.
Read the full abstract ▾
Computation and Language
Improving Diversity in LLM Short Story Generation
Zahra Solati Dehkordi and Vasileios Lampos
Large
language models (LLMs) can generate accurate responses, but these are void of diversity. We attempt to address this for the task of creative short story generation. Drawing on established writing conventions and known LLM limitations, we target variation in genre, tone, style, and named entities. To promote diversity across these dimensions, we introduce DivLM, an LLM post-training framework consisting of two phases. First, we perform continued pre-training on a creative writing corpus and restore instruction-following capabilities using weight residuals. We then apply reinforcement learning with a custom, composite reward function that jointly maximizes diversity across the targeted narrative dimensions while maintaining response quality. Our empirical results on two LLM families show that DivLM increases diversity metrics by more than 9% on average compared to alternative approaches, while preserving instruction following, overall response quality, and similarity to human outputs.
Read the full abstract ▾
Computation and Language
Domain adaptation of Russian ModernBERT for long legal documents
I. Litvak, D. Gvozdetsky, F. Lashkin et al. (+7 more)
We investigate whether continued pretraining on Russian legislative documents improves a Russian ModernBERT encoder on legal text. The adapted model, RuModernBERT-ruLaw, was trained on a corpus reported to contain 304,382 legislative documents and 194,425,905 corpus
tokens. Corpus
token counts are distinguished from positions produced by the model tokenizer. We compare the original and adapted encoders on a fixed external collection of 1,031 court-decision segments. Both models receive the same hidden positions in each of five masking realizations. At maximum input lengths of 512, 2,048, and 8,192
tokens, mean masked-
token cross-
entropy decreases by 0.10942, 0.07052, and 0.06604 natural-log units, respectively. The reported 95% intervals summarize sensitivity to masking on this fixed collection; they do not quantify uncertainty across document collections. A second evaluation addresses legal-entity extraction. The original and adapted models achieve entity-level F1 scores of 0.99852 and 0.99820. However, 99.95% of test spans have the same normalized surface form and class in the training split. This evaluation therefore provides limited evidence about transfer to previously unseen forms. The paper explains the masking objective, overlapping windows, averaging rules, and exact entity-boundary scoring using editable diagrams and clearly marked illustrative examples. The comparison supports lower masked-
token prediction loss for the studied pair of models and collection. It does not isolate the contribution of distant context or establish practical legal utility.
Read the full abstract ▾
All authors I. Litvak, D. Gvozdetsky, F. Lashkin, V. Kirova, S. Lagutin, V. Volf, T. Maksiyan, A. Kostin, R. Leva, I. Kiselev
Computation and Language
SAFE-MR: Evidence Sufficiency Learning for Selective Multimodal Rumor Detection
Shiwen Ni
Multimodal rumor detectors increasingly rely on retrieved evidence, yet relevant evidence is not necessarily sufficient for verification. Missing provenance, duplicated reports, and unresolved contradictions can produce confident predictions without adequate support. We introduce SAFE-MR, a framework that separates claim veracity from evidence sufficiency. The method decomposes image-text posts into verifiable claims, constructs a relation-aware claim-evidence graph, and aggregates evidence using provenance and contextual compatibility. Separate veracity and sufficiency heads support selective prediction, while evidence interventions encourage stability under irrelevant additions and sensitivity to evidence removal. On NewsCLIPpings, VERITE, and XFacta, SAFE-MR achieves macro-F1 scores of 91.2%, 75.8%, and 85.2%, respectively. Against the matched backbone with evidence, its macro-F1 gains are 2.2, 4.9, and 4.8 percentage points. On the diagnostic selection set, SAFE-MR reduces AURC from 0.105 for maximum-probability rejection to 0.075 and lowers error at 80% coverage from 13.8% to 8.5%. Evidence-perturbation and ablation results support the role of sufficiency learning and intervention training in improving selective verification.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.IR
Reading the Mood: Emotion-Guided Book-to-Music Recommendation via CGANs and LLMs
Manousos Linardakis and Georgios Alexandridis
Background music that matches the mood of a text has been shown to make readers feel more immersed and improve their reading experience, motivating recommender systems that pair books with mood-matched music. In this direction, we present Sentiment Aware Generative
Adversarial Network for Cross Domain Recommendation (SAGA-CDR), a two-phase cross-domain recommendation framework that personalizes music suggestions and emotionally aligns them with the book being read. In the first phase,
transformer-based sentiment
embeddings are constructed from user reviews and mapped across domains via a Conditional Generative
Adversarial Network, whose mask-conditioned generator handles missing sentiment components and injects stochasticity for richer preference transfer. A compact rating neural network then fuses sentiment-specific interaction scores with a collaborative filtering prior to predict music ratings. In the second phase, large
language models classify each book into a valence-arousal emotional quadrant, and candidate tracks are filtered to match that quadrant. Experiments on both the English Amazon and Chinese Douban
datasets show that SAGA-CDR achieves the best rating prediction accuracy on Amazon (RMSE 0.98) and the lowest RMSE on Douban (0.91), with ranking performance competitive with the strongest sentiment-aware baseline, even in cross-lingual settings.
Read the full abstract ▾
Computation and Language
MedPrune: Topology-Efficient Multimodal Multi-Agent Communication Evolution for Medical VQA Tasks
Jiuheng Wan, Runze Li, Chen Chen et al. (+5 more)
While medical
multimodal large
language models (Med-MLLMs) advance medical visual question answering (VQA), existing clinical workflow-inspired multi-
agent frameworks suffer from interaction patterns and excessive computational overhead caused by redundant communication
topologies. In this paper, we propose MedPrune, an efficient medical
multimodal multi-
agent collaboration framework that dynamically prunes both nodes and edges from the communication
topology to enhance
reasoning ability and
token efficiency. Specifically, we first formulate the diagnostic process as a heterogeneous communication graph, where nodes represent specialist
agents from various departments and edges capture intra- and inter-departmental interactions. Building on this graph, we introduce two sparsification mechanisms to enable adaptive collaborative
evolution: (1) Heterogeneous Node Sparsification, which eliminates task-irrelevant specialist
agents irrelevant to the current
multimodal question via reinforcement learning-driven topological optimization, and (2) Heterogeneous Edge Sparsification, which selectively retains only the most diagnostically salient intra- and inter-departmental connections by jointly optimizing task performance and topological complexity. Extensive medical VQA experiments under full-set and few-shot training settings prove MedPrune surpasses multi-
agent baselines and boosts
token efficiency with strong
adversarial robustness.
Read the full abstract ▾
All authors Jiuheng Wan, Runze Li, Chen Chen, Tingyuan Hu, Daiyang Yu, Yimin Jing, Taolin Zhang, Richang Hong
Computation and Language
Programmatic Search Agents: Extending Agentic Search Beyond Query Reformulation
Jiaming Qian, Huiyan Yang, Mandi Liu et al. (+7 more)
Search
agents adapt their queries, yet fixed search interfaces leave candidate processing and evidence presentation outside the
agent's direct control. Our trajectory analysis shows that supporting passages can be retrieved yet never delivered to the
agent; a same-page oracle intervention shows that changing the returned evidence can reduce subsequent search. We introduce Programmatic Search
Agent (PSA), which makes a local executable computation over candidates the unit of a search action. PSA unifies a persistent candidate workspace, flexible primitive composition, and selective evidence presentation. It incrementally generates program
cells that reuse candidates, execute dependent operations, and select what the
agent inspects next. The runtime resolves specified data dependencies within each
cell, while the
agent adapts its search strategy across
cells as new evidence arrives. We compare PSA with the Query-based
Agent and Tool-based
Agent on InfoSeek-Eval and BrowseComp-Plus using five policy backbones without task-specific training. All three interfaces share the search substrate, and the Tool-based
Agent also shares PSA's primitives and persistent workspace. Relative to the Query-based
Agent, PSA improves macro-averaged task success by 4.00 and 7.56 percentage points on the two
benchmarks, respectively; within-backbone reductions in final-step
tokens average 28.3% and 33.9%. These results support extending
agent control beyond query reformulation to the processing and presentation of retrieved evidence. Code will be released subject to approval.
Read the full abstract ▾
All authors Jiaming Qian, Huiyan Yang, Mandi Liu, Jie Liu, Wenkai Shen, Pengyang Zhou, Jing Jin, Jin Ma, Dezhi Ye, Chaochao Chen
Computation and Language · Cross-listed from cs.LG
Aligning Multimodal Patient Evidence with Biomedical Knowledge Graphs for Clinical LLMs
Jiawen Du, Arshan Ali Khan, Chenhao Zhang et al. (+7 more)
Clinical questions often depend on linking a patient's
multimodal evidence to external biomedical knowledge, yet existing predictive systems rarely represent such links explicitly, so they can neither be traced to their evidence sources nor removed to measure their contributions. We present MM-KG (
Multimodal Knowledge Graph), which represents heterogeneous,
multimodal patient observations and biomedical concepts as separate layers in one typed graph, joined by explicit
alignment edges. First, modality-specific harmonizers convert EHR text, imaging, genomic, and biospecimen data into typed observations mapped to UMLS concepts, which a route-prioritized aligner links to a biomedical knowledge graph. Query-conditioned
retrieval then selects a compact subgraph for downstream use by a large
language model or a graph neural network. We build MM-KGs for MIMIC-IV and ADNI, and evaluate them with a 2x2 design that separates patient evidence, biomedical knowledge, and their interaction. On questions that require both sources, neither source alone performs far above chance, whereas their combination yields a drug-controlled AUROC interaction of +0.194 on MIMIC and +0.299 on ADNI. On held-out five-candidate ranking, MM-KG outperforms MindMap by +0.131 Hits@1 and leads an adapted GraphCare on the items that require consulting the patient, and deleting the single answer-bearing relation from the retrieved packet returns Hits@1 to the no-knowledge baseline. Finally, query-conditioned
retrieval reaches 0.731 AUROC with 6.8x less context than the strongest generic policy, whereas static knowledge graph context gives no consistent gain on ordinary outcome prediction. Knowledge graphs thus benefit clinical LLMs not as background context but as explicit links between
multimodal patient evidence and the relation a question requires, and MM-KG makes these links retrievable, traceable, and testable.
Read the full abstract ▾
All authors Jiawen Du, Arshan Ali Khan, Chenhao Zhang, Zachary Plotkin, Li Shen, Qi Long, Yun Li, Can Chen, Tianlong Chen, Nicholas Konz
Computation and Language · Cross-listed from cs.LG
How Sparse Probability Maps Shape Mixture-of-Experts Routing
Tomás Brogueira, Marcos Treviso and Miguel Couceiro
Mixture-of-experts (MoE) routers typically apply softmax to the router scores and keep the top-K experts, making every
token use exactly K experts. Sparsity-inducing probability maps such as sparsemax, alpha-entmax and normmax can adaptively assign exact zeros to selected experts, and therefore appear to offer
token-dependent expert participation, even when using the same top-K machinery. In this work, we study whether and how this sparsity survives training. We train matched 300M and 1B top-2 MoE
language models with softmax, 1.5-entmax, sparsemax and 2-normmax, and find that the maps behave very differently once trained: at 1B, entmax discards 30% less probability mass than softmax while almost never dropping a selected expert, sparsemax retains the most mass, and normmax routes 21% of
tokens to a single expert. These outcomes are not properties of the maps alone. Each map drops a selected expert only when the gap between the two largest scores reaches a fixed threshold, and the trained routers differ in the score distribution they learn: the entmax router learns scores with roughly half the spread of softmax's, which keeps its top-2 gaps below its threshold, while sparsemax and normmax, which share the same threshold, learn different gap distributions and hence different participation. Routers thus co-adapt their scores to the map, and a map's capacity to produce zeros does not by itself determine expert participation. While none of the sparse maps improves validation loss over softmax, they make the trained models far less sensitive to selecting more experts at inference: sparsemax trained with K=2 loses 0.02 nats when run with K=8, where softmax loses 0.58. Our results indicate that adaptive MoE routing has to be designed around the joint behavior of the probability map and the learned scores, rather than around the map alone.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
The Pushback Paradox: A Two-Probe Diagnostic for Language Model Compliance
Stefan Bühler, David Exler, Markus Reischl et al. (+1 more)
Are
language models compliant with user instructions? A model that always complies can be stopped but also exploited, while one that always resists can be neither exploited nor stopped. We contribute an open two-probe
benchmark that can place any
language model on this spectrum. In the active probe, a user instructs the model to act and accept a lower payoff, which measures exploitability. In the passive probe, the user instructs it to wait and give up a higher payoff, which measures stoppability. The two compliance rates combine into a compliance index $\kappa$. Applied to twelve
language models, the
benchmark shows that seven mostly follow the instruction in both probes and justify their action by pointing to the instruction. Only Claude Sonnet-4.6 and Claude Opus-4.7 can be stopped without being exploitable, Claude Opus-4.6 and GPT-5-mini resist both instructions, and no model is exploitable but unstoppable. Knowing where a
language model sits on the compliance index $\kappa$ matters for human operators and for multi-
agent systems, whether distributed or orchestrated.
Read the full abstract ▾
All authors Stefan Bühler, David Exler, Markus Reischl, Mark Schutera
Computation and Language
Reward Stealing Attack on Large Language Models
Jiaming Qian, Pengyang Zhou, Jiahe Xu et al. (+1 more)
Adversarial attacks on Large
Language Models (LLMs) aim to induce harmful content. However, existing methods suffer from high computational costs or strict model-pairing dependencies, limiting their scalability and transferability. We propose Reward Stealing Attack (ReSA), an
adversarial attack framework that targets the latent safety reward underlying LLM
alignment. ReSA employs maximum
entropy inverse reinforcement learning to recover a proxy reward model solely from the aligned model's behavior. The extracted reward is then reversed at inference time to derive an
adversarial policy, efficiently implemented via a reward-guided decoding mechanism. Experiments demonstrate that a single recovered reward generalizes across
prompts and diverse models to reveal a fundamental
alignment vulnerability, enabling ReSA to significantly outperform existing attacks in effectiveness and transferability. The code is available at https://github.com/GarminQ/ReSA.
Read the full abstract ▾
All authors Jiaming Qian, Pengyang Zhou, Jiahe Xu, Chaochao Chen
Computation and Language · Cross-listed from cs.AI
Language models can notice an impossible engineering problem yet still report it as solved
Shaoliang Yang and Jun Wang
Language models draft engineering calculations, but answer accuracy does not show whether they reject an impossible problem. We tested 14 models on 30 pairs of mechanics problems, each with a valid version and one made impossible by changing a given value or assumption. Two independent solvers verified every answer key and showed that each flawed problem was physically impossible. We scored solving of valid problems separately from rejection of their flawed counterparts. Each reply required a "solved" or "cannot solve" status; rejection meant "cannot solve" or withholding an answer. The initial
prompts did not warn that problems could be flawed. Across three recent models, 12 of 90 replies failed to reject a flawed problem. In 11 of these replies, the model stated the flaw, answered a corrected problem and still reported the original as "solved", according to artificial intelligence raters and numerical checks. We later retested four models from one provider, offering "flawed" instead of "cannot solve" and asking them to name and explain the defect. Three models showed statistically significant increases in rejection, but valid-problem solving fell in three. Evaluations therefore need to score both versions and distinguish flaw recognition from the reported status.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
What Matters for Latent Reasoning with Flow Matching
Yassine Ouali, Adrian Bulat and Georgios Tzimiropoulos
Latent
reasoning lets a large
language model (LLM) think in a continuous space and verbalize only the answer. We argue that an effective latent thought must meet five requirements: it should be useful, helping produce the correct answer rather than merely changing it, diverse, so that resampling yields different
reasoning trajectories, explainable, so that a decoded chain of thought (CoT) reflects
reasoning the answer actually follows, refinable with more inference compute, and efficient, costing less than an explicit CoT at comparable accuracy. Current methods rarely meet these requirements: they learn shortcuts from the question, distill the explicit CoT into their weights, or imitate it one
token at a time. We focus on flow matching in a learned latent space, the family we argue is best placed to meet them, and identify the training choices that make it work. The result is Flow-based Latent
Reasoning (
FLaRe), a simple recipe covering what the latent space encodes and how to shape it, where to train the flow, how to read out the answer, and a final stage of training on the model's own verified thoughts. A probe for each requirement shows that
FLaRe improves on prior latent methods in all five. It also compares favorably with them on arithmetic
benchmarks, while reaching 97% of the accuracy of explicit CoT at a quarter of its latency.
Read the full abstract ▾
Computation and Language
Wikidata Search Traces: A Dataset for Training Knowledge Graph Search Agents
Mohamed Chenene, Carlos Rosas-Hinostroza, Anastasia Stasenko et al. (+2 more)
Wikidata is one of the largest open knowledge bases, yet answering a complex question over it still requires a SPARQL query that names the right entities and properties and chains their relations.
Language models offer a natural-
language alternative but answer largely from
memory, which is least reliable for less prominent entities. We study
agents that instead answer by exploring the graph, and argue that two obstacles limit them: the lack of training data recording how a solver explores, and interfaces that add large graph results directly to the model's context. We test three hypotheses: that the difficulty of graph search can be controlled through the structure of a question rather than only through obscure entities or wording; that much of the failure on long-horizon search comes from how retrieved evidence is managed rather than from the model itself; and that, in a suitable environment, open-weight models can match commercial closed ones. We construct multi-hop questions on a frozen Wikidata snapshot by replacing named entities with nested conditions, checking after each expansion that the target remains unique and that every new condition is necessary. We release 10,235 solving traces over single-entity and multi-hop questions, together with the recursive
language model (RLM) harness that produced them, in which models batch graph calls, keep results in persistent Python state and interpret selected evidence through sub-calls. On 100 questions, the harness improves both models we ran under both interfaces compared with direct tool calling over the same functions: gpt-6-luna rises from 49 to 61 correct answers, doubling its multi-hop accuracy, and Qwen3.8-27B, an open-weight model served on a single GPU, from 60 to 74.
Read the full abstract ▾
All authors Mohamed Chenene, Carlos Rosas-Hinostroza, Anastasia Stasenko, Shani Evenstein Sigalov, Pierre-Carl Langlais
Computation and Language
Representation-Space MMD for Diffusion Language Models
Ilya Drobyshevskiy, Ilia Sudakov, Maksim Semenov et al. (+7 more)
We introduce a post-training method for
diffusion language models (DLMs) that minimizes Maximum Mean Discrepancy (MMD) between generated and reference distributions in the feature space of a frozen pretrained DLM. To estimate MMD, we retain contextual features at individual
token positions, obtaining multiple observations per sequence from a single extractor pass. We optimize this objective using policy gradients for discrete models and direct differentiation through generated latents for continuous models. In both cases, computing the loss directly from these features enables efficient post-training without full sampling trajectories or jointly trained auxiliary models. Experiments show lower generative perplexity at comparable
entropy on OpenWebText and better accuracy-computation trade-offs on GSM8K. On 16B DMax-LLaDA2.0 models with hybrid masked-uniform
diffusion, we increase decoding parallelism with similar or higher accuracy on math and code
benchmarks.
Read the full abstract ▾
All authors Ilya Drobyshevskiy, Ilia Sudakov, Maksim Semenov, Denis Kuznedelev, Maksim Ignatov, Pavel Temirchev, Nikita Balagansky, Viacheslav Meshchaninov, Nikita Gushchin, Dmitry Baranchuk
Computation and Language
LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches
Shaokun Zhang, Yifan Zhang, Jian Hu et al. (+5 more)
Reinforcement learning has greatly advanced the capabilities of large
language models, but its
memory demands remain a barrier to broader adoption. We introduce LoGRA, an approach to RL post-training that reduces
memory by retaining useful learning signals in low-rank gradient sketches. These compact representations support both model updates and efficient policy synchronization. To prevent overly large updates from disrupting learning, we complement gradient compression with predicted-KL step control, which estimates policy changes before applying each update and adjusts its magnitude accordingly. With all techniques combined, LoGRA reduces average training
memory usage by up to 45.7% across
reasoning tasks without compromising performance. It also enables stable training of a 27B-parameter model for over 1,100 steps on a single eight-GPU node, where dense Adam runs out of
memory, making previously
memory-infeasible RL training practical. Code is available in the \href{https://github.com/skzhang1/labs-molt/tree/logra/examples/scripts/logra}{Molt library}.
Read the full abstract ▾
All authors Shaokun Zhang, Yifan Zhang, Jian Hu, Yueying Li, Hao Zhang, Binfeng Xu, Jan Kautz, Yi Dong