Computation and Language · Cross-listed from cs.CV
DentalGPT: Incentivizing Multimodal Complex Reasoning in Dentistry
Zhenyang Cai, Jiaming Zhang, Junjie Zhao et al. (+21 more)
Reliable interpretation of
multimodal data in dentistry is essential for automated oral healthcare, yet current
multimodal large
language models (MLLMs) struggle to capture fine-grained dental visual details and lack sufficient
reasoning ability for precise diagnosis. To address these limitations, we present DentalGPT, a specialized dental MLLM developed through high-quality domain knowledge injection and reinforcement learning. Specifically, the largest annotated
multimodal dataset for dentistry to date was constructed by aggregating over 120k dental images paired with detailed descriptions that highlight diagnostically relevant visual features, making it the
multimodal dataset with the most extensive collection of dental images to date. Training on this
dataset significantly enhances the MLLM's visual understanding of dental conditions, while the subsequent reinforcement learning stage further strengthens its capability for
multimodal complex
reasoning. Comprehensive evaluations on intraoral and panoramic
benchmarks, along with dental subsets of medical VQA
benchmarks, show that DentalGPT achieves superior performance in disease classification and dental VQA tasks, outperforming many state-of-the-art MLLMs despite having only 7B parameters. These results demonstrate that high-quality dental data combined with staged adaptation provides an effective pathway for building capable and domain-specialized dental MLLMs.
Read the full abstract ▾
All authors Zhenyang Cai, Jiaming Zhang, Junjie Zhao, Ziyi Zeng, Yanchao Li, Jingyi Liang, Junying Chen, Yunjin Yang, Jiajun You, Shuzhi Deng, Tongfei Wang, Wanting Chen, Chunxiu Hao, Ruiqi Xie, Zhenwei Wen, Xiangyi Feng, Zou Ting, Jin Zou Lin, Jianquan Li, Guangjun Yu, Liangyi Chen, Junwen Wang, Shan Jiang, Benyou Wang
Computation and Language · Cross-listed from cs.CV
HFS: Holistic Query-Aware Frame Selection for Efficient Video Understanding
Yiqing Yang, Yun Li, Daiqing Qi et al. (+5 more)
Key frame selection is essentially a set-level optimization problem: the quality of the selected subset depends on the interactions among frames, rather than the score of any single frame. Existing methods generally exhibit three major limitations. Point-wise methods score each frame independently and ignore inter-frame dependencies. Although the training-free set-level methods explicitly model the inter-frame relationships, their selection criteria are fixed and cannot be adapted through downstream task feedback. Learnable methods can leverage data-driven training; however, they lack an explicit, differentiable set-quality objective and rely on offline-generated supervision signals. To address these limitations, we propose an end-to-end trainable and task-adaptive framework for frame selection. A Chain-of-Thought
prompt conditions a Small
Language Model (SLM) to extract task-specific latent query vectors, which are combined with
multimodal features to enable dynamic, query-aware frame scoring. We further formulate a continuous set-level objective function that jointly accounts for relevance, coverage, and redundancy, enabling differentiable set-level optimization via Gumbel-TopK for selecting optimal frame combinations. Finally, we employ a student-teacher mutual learning strategy, in which the student selector (SLM) and teacher reasoner (MLLM) are trained to align their frame-importance distributions via KL divergence. Combined with cross-
entropy loss, this design enables fully end-to-end optimization, eliminating reliance on static pseudo-labels. Experiments across multiple
benchmarks, including
Video-MME, LongVideoBench, MLVU, and NExT-QA, demonstrate that our method significantly outperforms existing frame-selection approaches.
Read the full abstract ▾
All authors Yiqing Yang, Yun Li, Daiqing Qi, Lehan Yang, Tianlong Wang, Wenhao Zhang, Sheng Li, Kin-man Lam
Computation and Language
Does Less Hallucination Mean Less Creativity? An Empirical Investigation in LLMs
Mohor Banerjee, Nadya Yuki Wangsajaya, Syed Ali Redha Alsagoff et al. (+3 more)
Large
Language Models (LLMs) exhibit remarkable capabilities in natural
language understanding and
reasoning, but suffer from
hallucination: the generation of factually incorrect content. While numerous methods have been developed to reduce
hallucinations, their impact on creative generations remains unexplored. This gap is particularly critical for AI-assisted scientific discovery, which requires both factual accuracy and creative hypothesis generation. We investigate how three
hallucination-reduction techniques: Chain of Verification (CoVe), Decoding by Contrasting Layers (DoLa), and
Retrieval-Augmented Generation (RAG), affect creativity in LLMs. Evaluating multiple model families (LLaMA, Qwen, Mistral) at varying scales (1B - 70B parameters) on two creativity
benchmarks (NeoCoder and CS4), we find that these methods have opposing effects on divergent creativity. CoVe enhances divergent thinking, DoLa suppresses it, and RAG shows minimal impact. Our findings provide guidance for selecting appropriate
hallucination-reduction methods in scientific applications, where the balance between factual accuracy and creative exploration is crucial.
Read the full abstract ▾
All authors Mohor Banerjee, Nadya Yuki Wangsajaya, Syed Ali Redha Alsagoff, Min Sen Tan, Zachary Choy Kit Chun, Alvin Chan Guo Wei
Computation and Language
Building Patient Journeys in Hebrew: A Language Model for Clinical Timeline Extraction
Kai Golan Hashiloni, Brenda Kasabe Nokai, Michal Shevach et al. (+7 more)
We present a new Hebrew medical
language model designed to extract structured clinical timelines from electronic health records, enabling the construction of patient journeys. Our model is based on DictaBERT 2.0 and continually pre-trained on over five million de-identified hospital records. To evaluate its effectiveness, we introduce two new
datasets – one from internal medicine and emergency departments, and another from oncology – annotated for event temporal relations. Our results show that our model achieves strong performance on both
datasets. We also find that vocabulary adaptation improves
token efficiency and that de-identification does not compromise downstream performance, supporting
privacy-conscious model development. The model is made available for research use under ethical restrictions.
Read the full abstract ▾
All authors Kai Golan Hashiloni, Brenda Kasabe Nokai, Michal Shevach, Esthy Shemesh, Ronit Bartin, Anna Bergrin, Liran Harel, Nachum Dershowitz, Liat Nadai Arad, Kfir Bar
Computation and Language
Mistake Notebook Learning: Batch-Clustered Failures for Training-Free Agent Adaptation
Xuanbo Su, Yingfang Zhang, Hao Luo et al. (+2 more)
With the growing adoption of Large
Language Model (LLM)
agents in persistent, real-world roles, they naturally encounter continuous streams of tasks and inevitable failures. A key limitation, however, is their inability to systematically learn from these mistakes, forcing them to repeat identical errors in similar contexts. Unlike prior training-free methods that primarily store raw instance-level experience or focus on retrieving successful trajectories, we propose Mistake Notebook Learning (MNL), a novel
memory framework that enables
agents to self-curate generalizable guidance from batch-clustered failures. This mechanism allows
agents to distill shared error patterns into structured "mistake notes," updating an external
memory only when batch performance improves to ensure stability. To further amplify adaptability, we integrate MNL with test-time scaling, leveraging aggregated failure patterns to actively steer the search process away from known pitfalls. Experiments on mathematical
reasoning, Text-to-SQL, and interactive
agent benchmarks show that MNL achieves competitive performance compared to existing
memory mechanisms and in-context methods in both effectiveness and efficiency. These findings position structured mistake abstraction as a critical lever for robust
agent evolution, enabling continuous improvement without the cost of parameter updates. The code is available at https://github.com/Bairong-Xdynamics/MistakeNotebookLearning/tree/main.
Read the full abstract ▾
All authors Xuanbo Su, Yingfang Zhang, Hao Luo, Xiaoteng Liu, Leo Huang
Computation and Language · Cross-listed from cs.LG
Rethinking Expert Trajectory Utilization in LLM Post-training for Mathematical Reasoning
Bowen Ding, Yuhan Chen, Jiayang Lyv et al. (+9 more)
Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) dominate the post-training landscape for mathematical
reasoning, yet differ fundamentally in their reliance on expert trajectories. To understand the optimal way to harness these trajectories for maximizing performance, we propose the Plasticity-Ceiling Framework. This framework empirically grounds the post-training landscape by decomposing the final performance ceiling into the foundational SFT performance and the subsequent RL plasticity (i.e., the maximum improvement via RL). Through extensive benchmarking, we establish the Sequential SFT-then-RL pipeline as the superior standard, overcoming the stability and premature convergence deficits inherent in synchronized approaches. Furthermore, we derive precise scaling guidelines: (1) Transitioning to RL at the Stable or Mild Overfitting Regime of SFT maximizes the final ceiling by securing a robust SFT foundation with substantial RL plasticity; (2) Refuting the “Less is More” hypothesis in SFT-then-RL scaling, we demonstrate that Data Scale determines the primary post-training potential, while Trajectory Difficulty acts as a performance multiplier; and (3) The Minimum Validation Loss of SFT serves as a reliable indicator for selecting the expert trajectories that maximize the ultimate performance ceiling. Our findings provide actionable guidelines for extracting maximum value from expert trajectories.
Read the full abstract ▾
All authors Bowen Ding, Yuhan Chen, Jiayang Lyv, Jiyao Yuan, Qi Zhu, Shuangshuang Tian, Dantong Zhu, Futing Wang, Heyuan Deng, Fei Mi, Lifeng Shang, Tao Lin
Computation and Language
CLINIC: Evaluating Multilingual Trustworthiness in Language Models for Healthcare
Akash Ghosh, Srivarshinee Sridhar, Raghav Kaushik Ravi et al. (+3 more)
Integrating
language models (LMs) in healthcare systems holds great promise for improving medical workflows and decision-making. However, a critical barrier to their real-world adoption is the lack of reliable evaluation of their trustworthiness, especially in
multilingual healthcare settings. Existing LMs are predominantly trained in high-resource
languages, making them ill-equipped to handle the complexity and diversity of healthcare queries in mid- and low-resource
languages, posing significant challenges for deploying them in global healthcare contexts where linguistic diversity is key. In this work, we present CLINIC, a Comprehensive
Multilingual Benchmark to evaluate the trustworthiness of
language models in healthcare. CLINIC systematically
benchmarks LMs across five key dimensions of trustworthiness: truthfulness,
fairness, safety, robustness, and
privacy, operationalized through 18 diverse tasks, spanning 15
languages (covering all the major continents), and encompassing a wide array of critical healthcare topics like disease conditions, preventive actions, diagnostic tests, treatments, surgeries, and medications. Our extensive evaluation reveals that LMs struggle with factual correctness, demonstrate bias across demographic and linguistic groups, and are susceptible to
privacy breaches and
adversarial attacks. By highlighting these shortcomings, CLINIC lays the foundation for enhancing the global reach and safety of LMs in healthcare across diverse
languages.
Read the full abstract ▾
All authors Akash Ghosh, Srivarshinee Sridhar, Raghav Kaushik Ravi, Muhsin Muhsin, Sriparna Saha, Chirag Agarwal
Computation and Language · Cross-listed from cs.CE
Task-Specific Sparse Feature Masks for Molecular Toxicity Prediction with Chemical Language Models
Kwun Sy Lee, Jiawei Chen, Fuk Sheng Ford Chung et al. (+3 more)
Reliable in silico molecular toxicity prediction is a cornerstone of modern drug discovery, offering a scalable alternative to experimental screening. However, the black-box nature of state-of-the-art models remains a significant barrier to adoption, as high-stakes safety decisions demand verifiable structural insights alongside predictive performance. To address this, we propose a novel multi-task learning (MTL) framework designed to jointly enhance accuracy and interpretability. Our architecture integrates a shared chemical
language model with task-specific
attention modules. By imposing an L1 sparsity penalty on these modules, the framework is constrained to focus on a minimal set of salient molecular fragments for each distinct toxicity endpoint. The resulting framework is trained end-to-end and is readily adaptable to various
transformer-based backbones. Evaluated on the ClinTox, SIDER, and Tox21
benchmark datasets, our approach consistently outperforms both single-task and standard MTL baselines. Crucially, the sparse
attention weights provide chemically intuitive visualizations that reveal the specific fragments influencing predictions, thereby enhancing insight into the model's decision-making process.
Read the full abstract ▾
All authors Kwun Sy Lee, Jiawei Chen, Fuk Sheng Ford Chung, Tianyu Zhao, Zhenyuan Chen, Debby D. Wang
Computation and Language
Minimal Clips, Maximum Salience: Long Video Summarization via Key Moment Extraction
Galann Pennec, Zhengyuan Liu, Nicholas Asher et al. (+2 more)
Vision-
Language Models (VLMs) are able to process increasingly longer
videos. Yet, important visual information is easily lost throughout the entire context and missed by VLMs. Also, it is important to design tools that enable cost-effective analysis of lengthy
video content. In this paper, we propose a clip selection method that targets key
video moments to be included in a
multimodal summary. We divide the
video into short clips and generate compact visual descriptions of each using a lightweight
video captioning model. These are then passed to a large
language model (LLM), which selects the K clips containing the most relevant visual information for a
multimodal summary. We evaluate our approach on reference clips for the task, automatically derived from full human-annotated screenplays and summaries in the MovieSum
dataset. We further show that these reference clips (less than 6% of the movie) are sufficient to build a complete
multimodal summary of the movies in MovieSum. Using our clip selection method, we achieve a summarization performance close to that of these reference clips while capturing substantially more relevant
video information than random clip selection. Importantly, we maintain low computational cost by relying on a lightweight captioning model.
Read the full abstract ▾
All authors Galann Pennec, Zhengyuan Liu, Nicholas Asher, Philippe Muller, Nancy F. Chen
Computation and Language
Improving Translation Quality by Selecting Better Data for LLM Fine-Tuning: A Comparative Analysis
Felipe Ribeiro Fujita de Mello and Hideyuki Takada
We investigated the impact of data selection on machine translation fine-tuning for open LLMs. Using Japanese-English corpora, we compare five selectors: TF-IDF, COMET Kiwi, QuRate, FD-Score, and random selection, under controlled training conditions. We observed that semantic selectors consistently outperform lexical and geometry-based heuristics, and that even when the selected data differ by less than 3%, the impact on model performance is substantial, underscoring the sensitivity of fine-tuning to data quality.
Read the full abstract ▾
Computation and Language
Mining Legal Arguments to Study Judicial Formalism
Tomáš Koref, Lena Held, Mahammad Namazov et al. (+3 more)
Courts must justify their decisions, but systematically analyzing judicial
reasoning at scale remains difficult. This study tests claims about formalistic judging in Central and Eastern Europe (CEE) by developing automated methods to detect and classify judicial
reasoning in decisions of Czech Supreme Courts using state-of-the-art natural
language processing methods. We create the MADON
dataset of 272 decisions from two Czech Supreme Courts with expert annotations of 9,183 paragraphs with eight argument types and holistic formalism labels for supervised training and evaluation. Using a corpus of 300,511 Czech court decisions, we adapt
transformer LLMs to Czech legal domain through continued pretraining and we experiment with methods to address
dataset imbalance including asymmetric loss and class weighting. The best models can detect argumentative paragraphs (82.6% Bal-F1), classify traditional types of legal argument (77.5% Bal-F1), and classify decisions as formalistic/non-formalistic (83.8% Bal-F1). Our three-stage pipeline combining ModernBERT, Llama 3.1, and traditional feature-based machine learning achieves promising results for decision classification while reducing computational costs and increasing explainability. Empirically, we challenge prevailing narratives about CEE formalism. We demonstrate that legal argument mining enables promising judicial philosophy classification and highlight its potential for other important tasks in computational legal studies. Our methodology can be used across jurisdictions, and our entire pipeline,
datasets, guidelines, models, and source codes are available at https://github.com/trusthlt/madon.
Read the full abstract ▾
All authors Tomáš Koref, Lena Held, Mahammad Namazov, Harun Kumru, Yassine Thlija, Ivan Habernal
Computation and Language
qa-FLoRA: Data-free query-adaptive Fusion of LoRAs for LLMs
Shreya Shukla, Aditya Sriram, Milinda Kuppur Narayanaswamy et al. (+1 more)
The deployment of large
language models for specialized tasks often requires domain-specific parameter-efficient finetuning through Low-Rank Adaptation (LoRA) modules. However, effectively fusing these adapters to handle complex, multi-domain composite queries remains a critical challenge. Existing LoRA fusion approaches either use static weights, which assign equal relevance to each participating LoRA, or require data-intensive supervised training for every possible LoRA combination to obtain respective optimal fusion weights. We propose qa-FLoRA, a novel query-adaptive data-and-training-free method for LoRA fusion that dynamically computes layer-level fusion weights by measuring distributional divergence between the base model and respective adapters. Our approach eliminates the need for composite training data or domain-representative samples, making it readily applicable to existing adapter collections. Extensive experiments across nine
multilingual composite tasks spanning mathematics, coding, and medical domains, show that qa-FLoRA outperforms static fusion by ~5% with LLaMA-2 and ~6% with LLaMA-3, and the training-free baselines by ~7% with LLaMA-2 and ~10% with LLaMA-3, while significantly closing the gap with supervised baselines. Further, layer-level analysis of our fusion weights reveals interpretable fusion patterns, demonstrating the effectiveness of our approach for robust multi-domain adaptation.
Read the full abstract ▾
All authors Shreya Shukla, Aditya Sriram, Milinda Kuppur Narayanaswamy, Hiteshi Jain
Computation and Language
Unifying Dynamic Tool Creation and Cross-Task Experience Sharing through Cognitive Memory Architecture
Jiarun Liu, Shiyue Xu, Yang Li et al. (+3 more)
Large
Language Model
agents face fundamental challenges in adapting to novel tasks due to limitations in tool availability and experience reuse. Existing approaches either rely on predefined tools with limited coverage or build tools from scratch without leveraging past experiences, leading to inefficient exploration and suboptimal performance. We introduce SMITH (Shared
Memory Integrated Tool Hub), a unified cognitive architecture that seamlessly integrates dynamic tool creation with cross-task experience sharing through hierarchical
memory organization. SMITH organizes
agent memory into procedural, semantic, and episodic components, enabling systematic capability expansion while preserving successful execution patterns. Our approach formalizes tool creation as iterative code generation within controlled sandbox environments and experience sharing through episodic
memory retrieval with semantic similarity matching. We further propose a curriculum learning strategy based on
agent-ensemble difficulty re-estimation. Extensive experiments on the
GAIA benchmark demonstrate SMITH's effectiveness, achieving 81.8% Pass@1 accuracy and outperforming state-of-the-art baselines including Alita (75.2%) and Memento (70.9%). Our work establishes a foundation for building truly adaptive
agents that continuously evolve their capabilities through principled integration of tool creation and experience accumulation.
Read the full abstract ▾
All authors Jiarun Liu, Shiyue Xu, Yang Li, Shangkun Liu, Yongli Yu, Peng Cao
Computation and Language
LegalRikai: Open Benchmark – Benchmark for Complex Japanese Corporate Legal Tasks
Shogo Fujita, Yuji Naraki, Yiqing Zhu et al. (+1 more)
This paper introduces LegalRikai: Open
Benchmark, a new
benchmark comprising four complex tasks that emulate Japanese corporate legal practices. The
benchmark was created by legal professionals under the supervision of an attorney. This
benchmark has 100 samples that require long-form, structured outputs, and we evaluated them against multiple practical criteria. We conducted both human and automated evaluations using leading LLMs, including GPT-5, Gemini 2.5 Pro, and Claude Opus 4.1. Our human evaluation revealed that abstract instructions prompted unnecessary modifications, highlighting model weaknesses in document-level editing that were missed by conventional short-text tasks. Furthermore, our analysis reveals that automated evaluation aligns well with human judgment on criteria with clear linguistic grounding, and assessing structural consistency remains a challenge. The result demonstrates the utility of automated evaluation as a screening tool when expert availability is limited. We propose a
dataset evaluation framework to promote more practice-oriented research in the legal domain.
Read the full abstract ▾
All authors Shogo Fujita, Yuji Naraki, Yiqing Zhu, Shinsuke Mori
Computation and Language
CIP: A Plug-and-Play Causal Prompting Framework for Mitigating Hallucinations under Long-Context Noise
Qingsen Ma, Dianyun Wang, Ran Jing et al. (+2 more)
Large
language models often hallucinate when processing long and noisy
retrieval contexts because they rely on spurious correlations rather than genuine causal relationships. We propose CIP, a lightweight and plug-and-play causal prompting framework that mitigates
hallucinations at the input stage. CIP constructs a causal relation sequence among entities, actions, and events and injects it into the
prompt to guide
reasoning toward causally relevant evidence. Through causal intervention and counterfactual
reasoning, CIP suppresses non causal
reasoning paths, improving factual grounding and interpretability. Experiments across seven mainstream
language models, including GPT-4o, Gemini 2.0 Flash, and Llama 3.1, show that CIP consistently enhances
reasoning quality and reliability, achieving 2.6 points improvement in Attributable Rate, 0.38 improvement in Causal Consistency Score, and a fourfold increase in effective information density. API level profiling further shows that CIP accelerates contextual understanding and reduces end to end response latency by up to 55.1 percent. These results suggest that causal
reasoning may serve as a promising paradigm for improving the explainability, stability, and efficiency of large
language models.
Read the full abstract ▾
All authors Qingsen Ma, Dianyun Wang, Ran Jing, Yujun Sun, Zhenbo Xu
Computation and Language
AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
Kuan-Wei Lu, Ding-Yong Hong, Pangfeng Liu et al. (+1 more)
Large
language models (LLMs) have achieved remarkable performance across a wide range of tasks, but their increasing parameter sizes significantly slow down inference. Speculative decoding mitigates this issue by leveraging a smaller draft model to predict candidate
tokens, which are then verified by a larger target model. However, existing approaches often require additional training, extensive hyperparameter tuning, or prior analysis of models and tasks before deployment. In this paper, we propose Adaptive Speculative Decoding (AdaSD), a hyperparameter-free decoding scheme that dynamically adjusts generation length and acceptance criteria during inference. AdaSD introduces two adaptive components: one to determine when to stop candidate
token generation and the other to decide
token acceptance, updated in real time based on
token entropy and Jensen-Shannon distance. This approach eliminates the need for pre-analysis or fine-tuning and is compatible with off-the-shelf models. Experiments on
benchmark datasets demonstrate that AdaSD achieves up to 1.46x speedup over vanilla speculative decoding while limiting accuracy degradation to under 1.8%, making it a practical solution for efficient and adaptive LLM inference.
Read the full abstract ▾
All authors Kuan-Wei Lu, Ding-Yong Hong, Pangfeng Liu, Jan-Jan Wu
Computation and Language
When Actions Teach You to Think: Reasoning-Action Synergy via Reinforcement Learning in Conversational Agents
Mrinal Rawat, Arkajyoti Chakraborty, Neha Gupta et al. (+1 more)
Supervised fine-tuning (SFT) has emerged as one of the most effective ways to improve the performance of large
language models (LLMs) in downstream tasks. However, SFT can have difficulty generalizing when the underlying data distribution changes, even when the new data does not fall completely outside the training domain. Recent
reasoning-focused models such as o1 and R1 have demonstrated consistent gains over their non-
reasoning counterparts, highlighting the importance of
reasoning for improved generalization and reliability. However, collecting high-quality
reasoning traces for SFT remains challenging – annotations are costly, subjective, and difficult to scale. To address this limitation, we leverage Reinforcement Learning (RL) to enable models to learn
reasoning strategies directly from task outcomes. We propose a pipeline in which LLMs generate
reasoning steps that guide both the invocation of tools (e.g., function calls) and the final answer generation for conversational
agents. Our method employs Group Relative Policy Optimization (GRPO) with rewards designed around tool accuracy and answer correctness, allowing the model to iteratively refine its
reasoning and actions. Experimental results demonstrate that our approach improves both the quality of
reasoning and the precision of tool invocations, achieving a 1.5% relative improvement over the SFT model (trained without explicit thinking) and a 40% gain compared to the base of the vanilla Qwen3-1.7B model. These findings demonstrate the promise of unifying
reasoning and action learning through RL to build more capable and generalizable conversational
agents.
Read the full abstract ▾
All authors Mrinal Rawat, Arkajyoti Chakraborty, Neha Gupta, Roberto Pieraccini
Computation and Language
Leveraging LLMs for Title and Abstract Screening for Systematic Review: A Cost-Effective Dynamic Few-Shot Learning Approach
Yun-Chung Liu, Rui Yang, Jonathan Chong Kai Liew et al. (+4 more)
Systematic reviews are a key component of evidence-based medicine, playing a critical role in synthesizing existing research evidence and guiding clinical decisions. However, with the rapid growth of research publications, conducting systematic reviews has become increasingly burdensome, with title and abstract screening being one of the most time-consuming and resource-intensive steps. To mitigate this issue, we designed a two-stage dynamic few-shot learning (DFSL) approach aimed at improving the efficiency and performance of large
language models (LLMs) in the title and abstract screening task. Specifically, this approach first uses a low-cost LLM for initial screening, then re-evaluates low-confidence instances using a high-performance LLM, thereby enhancing screening performance while controlling computational costs. We evaluated this approach across 10 systematic reviews, and the results demonstrate its strong generalizability and cost-effectiveness, with potential to reduce manual screening burden and accelerate the systematic review process in practical applications.
Read the full abstract ▾
All authors Yun-Chung Liu, Rui Yang, Jonathan Chong Kai Liew, Ziran Yin, Henry Foote, Christopher J. Lindsell, Chuan Hong
Computation and Language
Multi-Intent Spoken Language Understanding: Methods, Trends, and Challenges
Di Wu, Ruiyu Fang, Liting Jiang et al. (+8 more)
Multi-intent spoken
language understanding (SLU) involves two tasks: multiple intent detection and slot filling, which jointly handle utterances containing more than one intent. Owing to this characteristic, which closely reflects real-world applications, the task has attracted increasing research
attention, and substantial progress has been achieved. However, there remains a lack of a comprehensive and systematic review of existing studies on multi-intent SLU. To this end, this paper presents a survey of recent advances in multi-intent SLU. We provide an in-depth overview of previous research from two perspectives: decoding paradigms and modeling approaches. On this basis, we further compare the performance of representative models and analyze their strengths and limitations. Finally, we discuss the current challenges and outline promising directions for future research. We hope this survey will offer valuable insights and serve as a useful reference for advancing research in multi-intent SLU.
Read the full abstract ▾
All authors Di Wu, Ruiyu Fang, Liting Jiang, Shuangyong Song, Xiaomeng Huang, Shiquan Wang, Zhongqiu Li, Lingling Shi, Mengjiao Bao, Yongxiang Li, Hao Huang
Computation and Language · Cross-listed from cs.LG
Adaptive Soft Rolling KV Freeze with Entropy-Guided Recovery: Sublinear Memory Growth for Efficient LLM Inference
Adilet Metinov, Gulida M. Kudakeeva, Bolotbek uulu Nursultan et al. (+1 more)
We present Adaptive Soft Rolling KV Freeze with
Entropy-Guided Recovery (ASR-KF-EGR), a training-free inference-time framework for efficient large
language model generation. Our method introduces a reversible soft-freeze mechanism that temporarily suspends key-value (KV) updates for low-importance
tokens identified within a sliding
attention window. Unlike eviction-based approaches that permanently discard context, ASR-KF-EGR preserves all
tokens in off-GPU storage and restores them on demand. We extend the framework with sublinear freeze scheduling, where freeze duration grows sublinearly with repeated low-importance detections, preventing over-aggressive compression. Preliminary experiments on LLaMA-3 8B demonstrate 55-67% reduction in active KV cache size while maintaining generation quality and passing needle-in-haystack
retrieval tests. The method is architecture-agnostic, requires no fine-tuning, and provides a practical solution for
memory-constrained deployment of long-context LLMs.
Read the full abstract ▾
All authors Adilet Metinov, Gulida M. Kudakeeva, Bolotbek uulu Nursultan, Gulnara D. Kabaeva
Computation and Language · Cross-listed from cs.AI
FutureWeaver: Planning Test-Time Compute for Multi-Agent Systems with Modularized Collaboration
Dongwon Jung, Peng Shi, Muhao Chen et al. (+1 more)
Scaling test-time computation has been shown to significantly improve large
language model (LLM) performance without additional training. However, extending these techniques to multi-
agent systems remains challenging: existing approaches lack principled mechanisms for allocating compute to enable effective collaboration, scaling coordination itself, or optimizing compute usage under explicit budget constraints. To address this gap, we propose FutureWeaver, a framework for planning and optimizing test-time compute allocation in multi-
agent systems under fixed budgets. It introduces collaboration modules, formalized as modular, callable functions that encapsulate reusable multi-
agent workflows and are automatically induced via self-play reflection from recurring interaction patterns. Building on these modules, it employs a dual-level planning architecture that jointly performs short-horizon action selection and long-horizon abstract lookahead to optimize inference trajectories under budget constraints. Experiments on complex
agent benchmarks demonstrate that FutureWeaver consistently outperforms baselines across diverse budget settings, validating its effectiveness for multi-
agent collaboration in inference-time optimization.
Read the full abstract ▾
All authors Dongwon Jung, Peng Shi, Muhao Chen, Yi Zhang
Computation and Language
SciLaD: A Large-Scale, Transparent, Reproducible Dataset for Natural Scientific Language Processing
Luca Foppiano, Sotaro Takeshita, Pedro Ortiz Suarez et al. (+6 more)
SciLaD is a novel, large-scale
dataset of scientific
language constructed entirely using open-source frameworks and publicly available data sources. It comprises a curated English split containing over 10 million scientific publications and a
multilingual, unfiltered TEI XML split including more than 35 million publications. We also publish the extensible pipeline for generating SciLaD. The
dataset construction and processing workflow demonstrates how open-source tools can enable large-scale, scientific data curation while maintaining high data quality. Finally, we pre-train a RoBERTa model on our
dataset and evaluate it across a comprehensive set of
benchmarks, achieving performance comparable to other scientific
language models of similar size, validating the quality and utility of SciLaD. We publish the
dataset and evaluation pipeline to promote reproducibility, transparency, and further research in natural scientific
language processing and understanding, including scholarly document processing.
Read the full abstract ▾
All authors Luca Foppiano, Sotaro Takeshita, Pedro Ortiz Suarez, Ekaterina Borisova, Raia Abu Ahmad, Malte Ostendorff, Fabio Barth, Julian Moreno-Schneider, Georg Rehm