Computation and Language
LoRA-Diffusion: Parameter-Efficient Fine-Tuning via Low-Rank Trajectory Decomposition
Iman Khazrak, Narges Nejad, Mohammadhossein Homaei et al. (+2 more)
Parameter-efficient fine-tuning methods such as LoRA have transformed the adaptation of large autoregressive
language models, enabling task-specific customization with substantially fewer trainable parameters. However, these methods have not been successfully extended to
diffusion-based
language models, which generate text through iterative denoising rather than sequential
token prediction. We propose LoRA-
Diffusion, a parameter-efficient fine-tuning approach that applies low-rank decomposition to the denoising trajectory instead of model weights. Unlike weight-based LoRA, which modifies individual transformation
matrices, our method learns low-rank perturbations to the entire
diffusion path from noise to output. We introduce trajectory-level low-rank adapters that modify each denoising step, step-adaptive rank allocation across
diffusion phases, and compositional multi-task learning that allows merging task-specific modules at inference without retraining. On SST-2, QNLI, and MRPC, we report
token-level denoising validation accuracy over five random seeds. LoRA-
Diffusion achieves the highest mean performance on SST-2 and strong performance on QNLI and MRPC. Joint multi-task training further shows that LoRA-
Diffusion achieves the highest
token-level accuracy among the evaluated methods. The approach reduces per-task storage compared with full fine-tuning and establishes a parameter-efficient fine-tuning framework for
diffusion language models.
Read the full abstract ▾
All authors Iman Khazrak, Narges Nejad, Mohammadhossein Homaei, Mostafa M. Rezaee, Robert C. Green II
Computation and Language
Comparative Analysis of Multilingual Pre-trained Models for Nepali Automatic Speech Recognition
Suman Paudel and Sarbin Sayami
Multilingual pretrained models nominally support Nepali, yet no controlled
benchmark has compared them under a single fine-tuning protocol. We fine-tune six pretrained models (XLSR-53, IndicWav2Vec, MMS-1B, Whisper-Medium, Whisper-Large-v3-Turbo, and Conformer-Hi) spanning CTC self-supervised, autoregressive encoder-decoder, and hybrid Conformer-CTC architectures, on the OpenSLR SLR54 Nepali corpus (~165 hours) using identical preprocessing, splits, optimizer, and family-matched learning-rate schedules. We evaluate Word Error Rate (WER), Character Error Rate (CER), and Real-Time Factor (RTF) on three independent test sets (OpenSLR, FLEURS, Common Voice). Whisper-Large-v3-Turbo (14.76% WER) and IndicWav2Vec (14.89% WER) tie at the top despite a 9x parameter gap and 40x pretraining-data gap, providing direct empirical evidence that
language-family proximity in pretraining can substitute for raw scale for in-domain Nepali. CTC decoders run up to 29x faster than autoregressive Whisper at the same accuracy, flipping the practical deployment preference toward CTC under any latency budget. Massively
multilingual pretraining (MMS-1B) yields the smallest out-of-domain degradation on FLEURS (+12.55 pp), indicating that scale buys robustness rather than peak in-domain accuracy. The resulting
benchmark provides the first standardized, multi-model, efficiency-aware reference numbers for Nepali ASR.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Beyond the Hivemind: Escaping LLM Homogeneity via Meta-Persona Anchoring and Sequential Temperature Scaling
Tairan Fu, Javier Conde, Carlos Arriaga et al. (+3 more)
Recent studies have identified an “Artificial Hivemind” effect in Large
Language Models (LLMs) causing models to converge on a narrow, homogenized consensus even for open questions. This semantic collapse limits the diversity of AI, resulting in high inter-response similarity ($\approx 0.80-0.90$) even under high-temperature sampling. In this paper, we propose a novel mitigation framework to increase diversity: Meta-Persona Anchoring combined with Filtered Temperature Scaling (FTS). Our approach utilizes a two-stage generation process: first, the model is prompted to self-select a unique, idiosyncratic persona to anchor its starting point; second, we apply a dual-stage sampling sieve, utilizing Top-$p$ filtering to preserve grammatical validity followed by extreme temperature scaling ($T \ge 4.0$) on the surviving candidates to explore the broadened probability distribution. We evaluate our method using the INFINITY-CHAT
dataset on state-of-the-art open weight models under $\sim$20B parameters. Our results demonstrate a significant reduction in semantic convergence, with average pairwise cosine similarity dropping from ($\approx 0.85$) to ($\approx 0.65$). Our scheme achieves a majority of questions below the 0.7 threshold, effectively reducing the gap between artificial mode collapse and human-level typological diversity. We provide our implementation as an open-source framework to enable more diverse and creative AI deployments.
Read the full abstract ▾
All authors Tairan Fu, Javier Conde, Carlos Arriaga, Gonzalo Martínez, Pedro Reviriego, Javier Coronado-Blázquez
Computation and Language · Cross-listed from cs.CV
Crayotter: Traceable Multi-Agent Workflows for Long-Form Video Editing
Lecheng Yan, Yichong Zhang, Xiantao Xu et al. (+12 more)
Long-form
video editing over heterogeneous footage requires
agents to coordinate source selection,
multimodal analysis, timeline construction, narration and subtitle
alignment, rendering, and revision while exposing intermediate state for inspection and repair. We present Crayotter, an open-source
multimodal multi-
agent demo system for
prompt-driven long-form
video editing. Crayotter organizes production around coverage-aware material preparation, artifact-grounded editing research, and tool-grounded timeline execution. Across these stages,
retrieval reports,
video analyses, editing blueprints, scheduler events, tool calls, intermediate renders, and final exports are treated as first-class artifacts rather than hidden transient state. The workbench supports local assets,
agent-assisted
retrieval, progress monitoring, artifact preview, failure diagnosis, interrupted-job resumption, and resource-aware asynchronous execution for long-running workflows. In a 23-theme evaluation, Crayotter achieves the highest human overall score (3.40/5) among the compared systems, with its largest margins in theme
alignment, narrative coherence, and editing smoothness. These results show that long-horizon
video editing
agents can be made traceable, inspectable, and practically controllable through observable production artifacts. Code, traces, and examples are publicly available at https://github.com/idwts/Crayotter.
Read the full abstract ▾
All authors Lecheng Yan, Yichong Zhang, Xiantao Xu, Jianze Lin, Ben Pan, Xiaoyu Zheng, Jiawei Qian, Anqi Wu, Jiahui Geng, Ruizhe Li, Fengyu Cai, Jingcheng Niu, Raymond Li, Wenxi Li, Chenyang Lyu
Computation and Language · Cross-listed from cs.CV
On the Limits of Token Reduction for Efficient Unified Vision Language Training
Siyi Chen, Weiming Zhuang, Jingtao Li et al. (+1 more)
Unified
vision-
language models (VLMs) integrate visual understanding and visual generation within a single autoregressive backbone, but their joint training is computationally expensive and largely overlooked from an efficiency perspective. In this work, we study the feasibility and limits of
token-reduction-based acceleration for unified VLM training. Through a systematic analysis of layerwise
attention allocation, we uncover a fundamental asymmetry: visual understanding exhibits substantial late-layer visual redundancy, whereas visual generation maintains persistent dependence on image
tokens across depth. Guided by this observation, we design task-specific accelerators that selectively reduce image-
token computation for each objective. While these methods achieve significant efficiency gains in isolated settings, we observe a consistent synergy loss under unified training – task-specific
token dropping necessitates divergent parameter pathways and eliminates the mutual performance gains typically observed in joint optimization. Our findings suggest that efficient unified modeling requires preserving shared cross-task structures, highlighting the need for synergy-aware acceleration strategies. Project page: https://chicychen.github.io/TokenReductionUnifiedVLM/.
Read the full abstract ▾
All authors Siyi Chen, Weiming Zhuang, Jingtao Li, Lingjuan Lv
Computation and Language
TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning
Yaxuan Kong, Qingren Yao, Yuqi Nie et al. (+7 more)
Time series data inform critical decisions across many real-world domains. While large
language model (LLM)
agents can analyze data through natural
language and tools, it remains unclear whether they can conduct reliable time series analysis across multi-turn conversations. Existing
benchmarks focus on single-step tasks such as
forecasting and anomaly detection, overlooking practical workflows where user goals evolve,
agents must build on prior analyses, and conclusions emerge from accumulated evidence. In this work, we introduce TimeSage-MT, a multi-turn
benchmark for agentic time series
reasoning with 240 tasks and 2,680 dialogue turns across 8 real-world domains, spanning basic exploration to decision-oriented analysis. TimeSage-MT is built through a reproducible pipeline that converts real-world time series data into multi-turn conversations with verifiable answers. It provides a unified evaluation protocol and public leaderboard for comparing time series agentic systems. To demonstrate the
benchmark's utility, we evaluate frontier LLMs alongside TimeSage, a novel structured
agent equipped with a comprehensive time series skill library. The results show sharp performance drops on decision-oriented tasks, driven by failures in
memory, uncertainty handling, and domain-based decision making. TimeSage-MT exposes critical gaps in current agentic
reasoning and provides a rigorous foundation for future development.
Read the full abstract ▾
All authors Yaxuan Kong, Qingren Yao, Yuqi Nie, Yichen Li, Yilei Shao, Stefan Zohren, Anna Vettoruzzo, Joaquin Vanschoren, Ming Jin, Qingsong Wen
Computation and Language · Cross-listed from cs.LG
CART: Context-Anchored Recurrent Transformer – A Parameter-Efficient Architecture with Learned Stability
Chad A. Capps
We present CART (Context-Anchored Recurrent
Transformer), a parameter-efficient
language model that reuses a single shared core block R times across depth. Unlike prior looped
transformers that recompute key-value tensors at every iteration, CART computes K and V once from a multi-layer prelude and has the recurrent core cross-attend to those frozen tensors via multi-head latent
attention. A learned Linear Time-Invariant (LTI) gate keeps the recurrence stable: its spectral radius settles in a narrow band (rho in [0.79, 0.83]) across all 36 fully-trained configurations. We evaluate CART on single consumer GPUs in two stages: a 64-configuration screen at 3,000 steps, then 36 configurations (P=6, R in {6,8,10}, three seeds) trained for 30,500 steps (~1B
tokens). Two patterns hold across widths d in {256,512,768,1024}: prelude depth P dominates loop count R, and the Stage-1 ranking of R reverses at full training (R=6 becomes best at d>=512). At the binding d=1024 parameter-parity test, CART does not beat a parameter-matched dense baseline, losing by 1-2% at stored-parameter parity and by ~10% at effective-parameter parity. Diagnostic ablations split the effective-parameter gap into ~5% from weight sharing and a residual ~5% from the heterogeneous prelude/anchor/core/coda framing; the recurrent-core machinery (hyper-connections, LTI gate, loop-index
embedding) is individually vestigial. Variable-R inference degrades on both sides of the trained R, a negative result for test-time depth scaling under this recipe.
Read the full abstract ▾
Computation and Language
Beyond Topical Similarity: Contrastive Evidence Retrieval with Interpretable Attention Alignment in RAG
Francielle Vargas, João Robiatti, Diego Alves et al. (+6 more)
Ensuring factuality and interpretability in RAG remains an open and urgent problem. We introduce Contrastive Evidence Rationale
Attention (CERA), the first
retrieval framework to employ subjectivity-based hard negative selection and inject an evidential inductive bias into contrastive learning through an auxiliary
attention alignment loss. CERA fine-tunes a dense retriever using two training objectives: triplet-based contrastive learning and interpretable
attention alignment, which supervises CLS-to-
token attention using a part-of-
speech-weighted masking distribution over human-annotated factual rationales as evidence signals. Experiments on a large corpus of clinical trial reports demonstrate that the subjectivity-based hard negative selection substantially improves
retrieval effectiveness compared to both Contriever and hard negative selection baselines. Furthermore, rationale
alignment improves faithfulness while maintaining competitive
retrieval performance, supporting the hypothesis that
attention can serve as a more faithful explanation of model behavior when guided by human rationales. Moving beyond topical similarity, CERA enables the retriever to identify the specific
tokens that constitute supporting evidence, promoting more interpretable evidence selection in RAG systems.
Read the full abstract ▾
All authors Francielle Vargas, João Robiatti, Diego Alves, Lucas Pascotti Valem, Maximilian Seeth, Sebastián Ferrada, Ameeta Agrawal, Daniel Pedronette, André Freitas
Computation and Language
Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech
Hongfei Du, Jiacheng Shi, Sidi Lu et al. (+2 more)
Integrating large
language models (LLMs) into text-to-
speech (TTS) systems has improved
speech expressiveness, yet interpretable emotional control remains challenging. Existing approaches primarily rely on external conditioning or global activation steering, offering limited insight into the internal representations underlying emotional control. In this work, we analyze
emotion-related variation in the semantic hidden states of LLM-based TTS models using sparse autoencoders (SAEs) to identify sparse latent features. Our analysis shows that emotional variation is distributed across multiple sparse latent features, while intervening on a small subset enables interpretable
emotion control. Building on this observation, we introduce a feature-level intervention framework for bidirectional
emotion induction and suppression without modifying backbone parameters. We further show that distinct latent features are associated with specific acoustic attributes (e.g., pitch), suggesting that emotional expression arises from coordinated latent contributions rather than a single global shift. Empirically, steering these sparse latent features achieves comparable or superior
emotion induction and suppression performance relative to global steering and existing TTS baselines.
Read the full abstract ▾
All authors Hongfei Du, Jiacheng Shi, Sidi Lu, Gang Zhou, Ye Gao
Computation and Language · Cross-listed from cs.LG
OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification
Yuhang Zhou, Lizhu Zhang, Yifan Wu et al. (+5 more)
On-Policy
Distillation (OPD) trains a student model on its own generative trajectories under dense
token-level feedback from a stronger teacher, mitigating both the off-policy distribution shift of Supervised Fine-Tuning (SFT) and the sparse credit assignment of Reinforcement Learning (RL). However, standard OPD faces two coupled limitations. First, it requires direct access to the teacher's
token-level logits, excluding a broad class of capable proprietary models from serving as teachers. Second, the
token-level logit signal itself is brittle, depending on a narrow overlap of plausible next
tokens between teacher and student, and prone to amplifying degenerate patterns such as repetition loops. In this paper, we introduce OmniOPD, a novel framework that addresses both limitations through a logit-free, chunk-level supervision signal. OmniOPD replaces deterministic logit matching with Monte Carlo rollouts that approximate the teacher's local preferences through a continuous semantic similarity metric over multi-
token chunks, and concentrates this supervision via a peak-
entropy scheduler that audits the student only at its high-uncertainty
reasoning forks. A Dirichlet-Multinomial
Bayesian prior and a base-model KL anchor further bound the variance of discrete sampling and prevent policy collapse across unaudited
tokens. Across competitive
benchmarks, OmniOPD surpasses the standard OPD approach by up to +28.64% on math, confirming that chunk-level semantic verification extracts a more reliable learning signal than
token-level logit matching, whose high information density is offset by significant noise and brittleness. Furthermore, when paired with stronger black-box teachers such as Claude-4.5-Haiku and Gemini-2.5-Flash, OmniOPD achieves an additional +9.54% relative on math over its open-weight teacher counterpart, advancing the student past the performance of self-exploratory RL.
Read the full abstract ▾
All authors Yuhang Zhou, Lizhu Zhang, Yifan Wu, Mingyi Wang, Bo Peng, Jiayi Liu, Xiangjun Fan, Zhuokai Zhao
Computation and Language
Peacemaker at ATE-IT: Automatic term extraction from Italian text for waste management data using encoder model
Mahdi Bakhtiyarzadeh, Hadi Bayrami Asl Tekanlou and Jafar Razmara
The development of automatic term extraction has become increasingly important in modern technology. Automatic term extraction can be found in virtually every search engine that is currently available to users. Recent advancements have provided promising results for the extraction of automatic terms; however, accurate labeling is difficult because of several factors, such as the limited number of annotated documents available for training and the complexity of extracting multi-word expressions due to shifts in the domain. In this paper, we will present a low-cost and interpretable method of automatic term extraction, developed specifically for Task A of the ATE Shared Task. This new method utilizes fine-tuning extraction strategies that can run on a small amount of computational resources. We evaluated our automated system using both type-level and micro-level measures of precision, recall, and F1-score to measure both complementary aspects of the extraction performance. According to the experimental results, our proposed approach achieves consistent and balanced performance compared to other teams. Even though the technique itself is relatively straightforward, it serves as a good starting point for low-resource models. Overall, the findings point toward the possibility of significant future advancements (in model expansion) with higher-level performance still able to retain their ability to be interpreted.
Read the full abstract ▾
Computation and Language
Ahmed Elhady, Eneko Agirre and Mikel Artetxe
Despite expanding their
multilingual coverage, the advanced
reasoning capabilities of LLMs remain largely confined to a few high-resource
languages like English. To address this, we propose an unsupervised Reinforcement Learning (RL) approach to enhance
multilingual reasoning by enforcing cross-lingual self-consistency: the principle that a model should produce the same final answer for equivalent problems in different
languages. Existing methods are limited by the scarcity of
multilingual reasoning data and show weak generalization to unseen
languages. Our approach requires neither gold answers nor parallel data, and it achieves average gains of up to 21.7% on MGSM across 10
languages. In addition, our method demonstrates strong generalization, with an 18.2% mean improvement on MGSM
languages unseen during training, and up to 6.2% gain on 3 out-of-distribution
benchmarks. These results show the potential of consistency-based methods to improve the
multilingual capabilities of LLMs without requiring supervised data.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Models
Mingzhong Sun, Teresa Yeo, Armando Solar-Lezama et al. (+1 more)
Studies of human
reasoning have shown that people are typically stronger at evaluating
reasoning than producing it from scratch. In contrast, large
reasoning models (LRMs) are trained to excel at producing long chains of
reasoning to solve complex problems. How then do LRMs perform at evaluating reasons? We investigate this with the Valid-Answer-Invalid-
Reasoning (VAIR)
dataset: math problems and solutions with trivial
reasoning flaws but valid answers, designed to isolate
reasoning evaluation from the confound of
reasoning production. Unlike humans, who we find are only 6% worse at grading than solving such problems, we find a substantial production-evaluation gap in LRMs: frontier models score as low as 48% when evaluating VAIR solutions, despite near-perfect solution production. Why this enigma? Through chain-of-thought (CoT) analysis, we find evidence of an answer confirmation bias: LRMs often produce then check for the correct answer instead of carefully verifying each step, fabricating rationalizations even when noticing anomalous
reasoning. Linear probes corroborate this, showing that while LRM activations encode some representation of valid
reasoning, they fail to robustly represent VAIR solutions as invalid. Causal patching of the final answer's representations causes LRM verdicts and activations to flip, demonstrating that answer validity is responsible for models' confirmation biases. These findings indicate an outstanding limitation in dominant approaches to
reasoning training, which incentivize LRMs to produce and confirm
reasoning towards correct answers, but not to robustly evaluate the underlying reasons.
Read the full abstract ▾
All authors Mingzhong Sun, Teresa Yeo, Armando Solar-Lezama, Tan Zhi-Xuan
Computation and Language · Cross-listed from cs.LG
Truthful AI Advisors: A Pre-Specified Benchmark for Large Language Model Honesty Under Preference Misalignment
Hamidreza Hasani Balyani, Seyed Pouyan Mousavi Davoudi, Alireza Amiri-Margavi et al. (+2 more)
Large
language models are increasingly deployed as advisors whose objective is not aligned with the user's: recommenders optimize for engagement, sales assistants for purchases. Whether they stay truthful when honesty conflicts with their own payoff is a core
alignment question. We turn the canonical Crawford-Sobel cheap-talk model into a pre-specified
benchmark for LLM honesty under preference misalignment, in which theory supplies an exact oracle. A sender observes a state omega in [0,1], wants the receiver's action near omega+b, and sends one costless message to a receiver whose ideal action is omega. For the positive-bias grid b in {0.01,0.04,0.08,0.12} the exact most-informative partition sizes are 7,4,3,2, with oracle normalized mutual information 0.5294, 0.3268, 0.2205, 0.1829. Extending a pre-registered 4-model run of 12,000 sender calls to eight models across two capability tiers and 39,569 logged calls, all models over-reveal relative to the most-informative
equilibrium by 1.8 to 4.5x: pooled normalized mutual information stays at 0.82-0.96 where the oracle prescribes 0.18-0.53. Informativeness declines with bias as predicted (beta = -1.71, t = -7.50) but never approaches the strategic optimum; rather than coarse partitions, models show near-full revelation with a constant upward offset tracking their bias (linear exaggeration). A structural hint separates capability from propensity: told the
equilibrium partition size,
reasoning models state a correct Crawford-Sobel
cell in 0.20-0.99 of messages while the non-
reasoning tier never exceeds 0.005. The capability is present but goes unexercised unless asked for, locating the failure in propensity rather than competence. A decoder ablation shows the finding is recoverable only when the receiver reads the sender's stated number: an
embedding-only decoder mis-reads the same data as near-babbling.
Read the full abstract ▾
All authors Hamidreza Hasani Balyani, Seyed Pouyan Mousavi Davoudi, Alireza Amiri-Margavi, Amin Gholami Davodi, Arshia Gharagozlou
Computation and Language
Before and After Temperature: A Distributional View of Creative LLM Generation
V. S. Raghu Parupudi, Harsha Ponnada, Aditi Kaushal et al. (+3 more)
Reference-free evaluation of large
language model (LLM) creativity relies on perplexity,
entropy, and top-1 margin. We show that a much stronger signal lives one step earlier in the pipeline: in how sampling temperature reshapes the model's
token distribution before the next
token is drawn. On Llama-3.1-8B-Instruct generations of 500 open-ended creative
prompts at $T \in \{0.3, 0.8, 1.5\}$, a single per-
token feature derived from this reshaping predicts the within-
prompt creativity rank at Spearman $\rho{=}0.918$ against an averaged gpt-4o\,/\,gemini-2.5-pro judge ($n{=}500$) and $\rho{=}0.870$ against a three-rater human-majority ranking ($n{=}150$). Each of four standard reference-free baselines (self-perplexity, mean predictive
entropy, top-1 margin, gzip compression ratio) tops out at $|\rho|\!\approx\!0.76$ on both ground truths: a gap of $+0.165$ on averaged-LLM and $+0.110$ on human-majority, both far larger than the spread among the baselines themselves. The two ground-truth panels agree with each other at $\rho{=}0.83$, above the inter-human ceiling of $\rho{=}0.77$, so the comparison is not bottlenecked by judge noise. Mechanistically, the win comes from a sharp distributional signature of the incoherence regime: at $T{=}1.5$ the cumulative-mass width $n_{95}(q)$ inflates from $\sim\!1$ to ${\sim}\!131$
tokens and post-temperature mass leaks off the pre-temperature top-$90\%$ plausible set by about $13$ percentage points. The per-
token aggregates do not separate $T{=}0.8$ from $T{=}0.3$; discriminating the two coherent regimes is left to sequence-level features.
Read the full abstract ▾
All authors V. S. Raghu Parupudi, Harsha Ponnada, Aditi Kaushal, S. Shria Parupudi, Saiteja Dasari, Sahiti Bulusu
Computation and Language · Cross-listed from cs.AI
Self-Revising Discovery Systems for Science: A Categorical Framework for Agentic Artificial Intelligence
Fiona Y. Wang and Markus J. Buehler
Scientific discovery is not only answer generation but revision of the representational regime in which evidence, artifacts, operations, and verifiers are typed. We develop a category-theoretic account of agentic discovery for materials science. In a fixed regime b with schema category S_b, the system state is a copresheaf I_t: S_b -> Set, and provenance is the category of elements \int_{S_b} I_t. Fixed-regime operation is an update on such states, endofunctorial only when provenance-preserving refinements are specified and preserved. Discovery is instead a verified regime transition u: S_b -> S_b': old artifacts are preserved, transported by the left Kan extension Lan_u I_t, and compared with the post-transition state to identify residual content beyond functorial transport. This separates
retrieval, search, and discovery without subjective novelty. We instantiate the framework in two systems. In Builder/Breaker, a
protein-mechanics world model is revised under a Minimum Description Length gate; the accepted law expresses within-chain flexibility as all-mode elastic compliance conditioned by slow collective-mode participation, or mode-conditioned compliance. In CategoryScienceClaw, typed skills, artifacts, open needs, workflow mutation, gates, stress tests, and public discourse become a proof-carrying knowledge-computation graph. A fiber-network example records candidate models, rejected alternatives, an AIC gate, perturbation tests, and an accepted orientation-tensor anisotropic stiffness surrogate over an isotropic fiber-count descriptor. Together, the cases show how category theory can be both a mathematical
language for discovery and an engineering specification for self-revising AI discovery systems.
Read the full abstract ▾
Computation and Language
Learning from Saturated Data: Signals Beyond Correctness for LLM Training
Hanno Hiss, Jasper Dekoninck and Martin Vechev
The growing capabilities of large
language models (LLMs) have led to the saturation of many
benchmarks and training
datasets used to improve them. Motivated by this, we investigate whether questions solved with perfect empirical accuracy can nevertheless be used to improve downstream performance. To do so, we replace binary correctness with two sources of more fine-grained quality signals: (1) pairwise LLM self-judgments, in which the model evaluates the relative quality of its own solutions, and (2)
token-level
entropy, where
token-level uncertainty is used as a proxy for solution quality. We incorporate these signals into several training
algorithms and evaluate them on Qwen3-1.7B-Base. When training exclusively on a simple arithmetic task, quality-based signals improve performance by up to $18.6\%$ over the base model, substantially outperforming SFT. On GSM8K, however, gains are more modest and depend strongly on the quality signal. For instance, self-judgments show poor agreement with a stronger external judge and can even degrade performance below the base model. Overall, our results suggest that quality-based training can extract useful signal from saturated questions for base models, but that applying such signals to more complex tasks requires careful calibration and further study.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Reliable Post-Retrieval Assembly for Agent Memory: Separating Evidence Extraction from Policy Execution
Vikas Reddy and Sumanth Reddy Challaram
LLM-based
memory systems can retrieve relevant evidence yet still fail when answer generation entangles semantic filtering, conflict resolution, prior suppression, and output generation in one step. We study this failure as a problem of post-
retrieval assembly. In the MemoryAgentBench (MAB) release used here, FactConsolidation explicitly states that newer facts have larger serial numbers, yet the best reported
retrieval/
memory result is 54% single-hop and all 22 reported systems score at most 7% multi-hop. We evaluate a structured assembly interface in which an LLM first extracts semantically matching evidence into a candidate representation and a separate stage executes the required answer policy. At 262K, this pipeline reaches 82%/93% single-hop and 27%/41% multi-hop with gpt-4o-mini/gpt-4o, exceeding every result reported in the MAB v3 FactConsolidation comparison. This is a task-level result, not a claim that the evaluated
memory architectures are broadly inferior. A controlled whole-pipeline comparison, with identical backbone, retrieved top-10 evidence, chunking, and n=100 per
cell, improves single-hop accuracy by 10.8 percentage points (pp) on average and 21 pp at 262K. A targeted comparison using the same extraction setup shows that changing only the final policy executor contributes 2.0 pp on average and 0 pp at 262K. Most of the gain therefore comes from separating evidence identification from final policy execution rather than from the freshness operator itself. A LongMemEval check finds no significant overall advantage (26/45 versus 29/45; paired exact McNemar p=0.45), bounding the result to current-value questions with explicit version metadata. The evidence identifies post-
retrieval assembly as a distinct reliability boundary between
retrieval and answer generation.
Read the full abstract ▾
Computation and Language
DrugClaw and DrugAudit: A Primary-Source-Grounded Agent and Authority-Aware Benchmark for Drug-Information Question Answering
Qing Wang, Bo Li, Jialu Liang et al. (+3 more)
Drug-information question answering is a high-stakes setting where hallucinated facts can mislead clinical decision-making and the provenance of each cited fact matters as much as the fact itself. We present DrugClaw, a multi-
agent retrieval-augmented system that queries a registry of drug and pharmacovigilance skills via a reflection-driven state-machine workflow and returns answers grounded in primary regulatory or peer-reviewed records. We also contribute DrugAudit, a 3,772-item authority-aware
benchmark with an evaluation panel that scores upstream-of-gold source match,
token-level semantic snippet overlap, and citation faithfulness under a dual-judge LLM-as-judge protocol with inter-judge kappa = 0.88 (almost-perfect). Across DrugAudit plus drug-related subsets of MedQA (751) and PubMedQA (512), DrugClaw is top-1 on every column of the headline table: composite Evidence Index under both judges, judge-mediated answer correctness, primary-source rate (0.918, +10.1 pp over next-best), faithfulness (0.887, +5.9 pp), MedQA (0.920), and PubMedQA (0.693).
Read the full abstract ▾
All authors Qing Wang, Bo Li, Jialu Liang, Daling Shi, Bob Zhang, Qianqian Song
Computation and Language
Consistent and Distinctive: LLM Benchmark Efficiency via Maximum Independent Set Prompt Selection on Similarity Graphs
Denica Kjorvezir, Marko Djukanović, Ana Gjorgjevikj et al. (+2 more)
Evaluating large
language models (LLMs) across comprehensive
benchmarks is expensive and time-consuming. We propose a graph-based
prompt selection framework that models each
benchmark as a similarity graph – nodes are
prompts connected if their
embedding-space distance falls above a configurable threshold – and applies Maximum Independent Set (MIS)
algorithms to select a maximally diverse, non-redundant subset. We evaluate four MIS solvers (CPLEX, GREEDY, Online-MIS, ReduMIS) across six
embedding models, three distance measures, six percentile thresholds, and four
benchmarks (GPQA, IFEval, MMLU-Pro, Omni-MATH) covering 66 LLMs. Our central hypothesis – that repeated selection under different random seeds yields consistent LLM rankings that may also differ from the full-
benchmark baseline – is strongly confirmed: Kendall's $W \geq 0.90$ in 99.2% of
stochastic configurations (mean $W = 0.997 \pm 0.008$), while at higher percentile thresholds selected subsets achieve 25–48%
prompt reduction on average. Ranking divergence from the full
benchmark ($\rho < 0.95$) occurs in only 15.95% of configurations, concentrated at low thresholds ($p_{10}$–$p_{20}$) and
benchmarks (GPQA, IFEval), identifying overly dense graphs as the primary failure mode.
Read the full abstract ▾
All authors Denica Kjorvezir, Marko Djukanović, Ana Gjorgjevikj, Gjorgjina Cenikj, Tome Eftimov
Computation and Language
UniD$^3$: A Knowledge Graph-Enhanced RAG Framework for Drug-Disease Discovery and Reasoning
Qing Wang, Tianshi Liu, Minghao Zhou et al. (+5 more)
Systematic characterization of drug-disease relationships is essential for drug discovery and repurposing, yet is hindered by the heterogeneity and rapid growth of biomedical literature. Existing
datasets rely on labor-intensive curation and are often incomplete, while LLM-only approaches suffer from
hallucination and weak evidence grounding. We introduce UniD$^3$, a unified framework that integrates Large
Language Models with Knowledge Graph-enhanced
Retrieval-Augmented Generation (KG-RAG) to extract, organize, and validate drug-disease knowledge across Drug-Disease Matching (DDM), Drug Effectiveness Assessment (DEA), and Drug-Target Analysis (DTA). UniD$^3$ processes 157,849 PubMed articles with Llama 3.3-70B and constructs knowledge graphs via a dual-stage strategy combining paper-level extraction with KG-level consolidation centered on drug and disease entities. These graphs support KG-RAG-based generation of structured
datasets, evaluated through external
benchmarks, fuzzy matching with curated resources, and clinician review. UniD$^3$ produces six knowledge graphs and large-scale
datasets, including 28,915 DDM, 15,042 DEA, and over 4,000 DTA QA pairs. External validation shows strong performance (F1: 0.85-0.87 for DDM/DEA; 0.82 for DTA), with clinician review confirming high reliability (AUROC = 0.90). KG-RAG-augmented models outperform standalone LLMs, and the UniD$^3$ chatbot enables interpretable, citation-supported exploration of drug-disease relationships. UniD$^3$ provides a scalable, extensible framework for transforming unstructured biomedical literature into high-quality, structured drug-disease knowledge, supporting AI-driven discovery, repurposing, and precision medicine.
Read the full abstract ▾
All authors Qing Wang, Tianshi Liu, Minghao Zhou, Jialu Liang, Sen Guo, Guangyu Wang, Jing Su, Qianqian Song
Computation and Language
Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing
Minglai Yang, Xinyan Velocity Yu, Pengyuan Li et al. (+22 more)
Document parsing and recognition are fundamental capabilities for
vision-
language models (VLMs) and document processing systems. However, existing Optical Character Recognition (OCR) and document parsing
benchmarks are increasingly limited in coverage and difficulty: many focus on common document genres or uniformly sampled pages where modern parsers already perform strongly, while offering limited annotation for expert-domain structures such as chemical formula, music notation, complex tables, and cross-page layouts. We introduce Dr. DocBench, a difficulty-aware
benchmark for expert-level document parsing. Built from a large-scale
multilingual book corpus, Dr. DocBench spans 52 BISAC subject domains and selects challenging documents through parser-failure-based sampling, targeting cases where multiple state-of-the-art systems struggle. It contains 4,514 annotated pages from long documents averaging around 100 pages, with 65k high-quality page- and block-level annotations for layout, reading order, hierarchical relations, and domain-specific visual contents. Evaluations of pipeline-based parsers and general-purpose VLMs show that strong performance on existing
benchmarks does not transfer to our expert-level document parsing. Our analysis reveals substantial failures across subjects, content types, and structural attributes, highlighting Dr. DocBench as a comprehensive testbed for diagnosing and advancing document intelligence.
Read the full abstract ▾
All authors Minglai Yang, Xinyan Velocity Yu, Pengyuan Li, Xinyu Guo, Zhenting Qi, Konwoo Kim, Longtian Ye, Xiaolong Luo, Jinhe Bi, Henry Zhang, Haris Riaz, Xuan Zhang, Yunze Xiao, Bangya Liu, Tom Tang, Yunfei Zhao, Qunshu Lin, Zihan Wang, Minghao Liu, Michael Lingzhi Li, Yilun Du, Jesse Thomason, Rogerio Feris, Alex Pentland, Zexue He
Computation and Language · Cross-listed from cs.AI
GuidaPA: Privacy-Preserving Chatbot for Public Administration via Federated Learning
Daniel M. Jimenez-Gutierrez, Albenzio Cirillo, Raffaele Nicolussi et al. (+2 more)
We present GuidaPA, a
privacy-preserving chatbot for the Italian Public Administration (PA) trained via
Federated Learning (FL) on documentation from two national PA platforms, SIGESON and SIDFORS. Our corpus includes approximately 8 pages of SIGESON manuals and 31 pages of SIDFORS manuals/FAQs; while this study uses public documentation as a safe proxy, the intended deployment extends to restricted internal sources (e.g., tickets, officer manuals, database extracts) that can not be centrally pooled due to regulatory and organizational constraints. GuidaPA integrates role-based access control, secure client-side preprocessing, explicit monitoring of non-IID effects, and parameter-efficient
federated fine-tuning of large
language models. Using QLoRA (4-bit) over 15
federated rounds with an 80/20 train-test split per client, we evaluate answer quality with ROUGE, BLEU-4, and METEOR. The best
federated model achieves ROUGE-1/2/L of 61.10/55.77/59.44, BLEU-4 of 45.02, and METEOR of 63.94-close to private centralized fine-tuning while keeping data on-site. Compared to the general-purpose baseline, domain fine-tuning improves ROUGE-1 from 41.45 to 62.18 and BLEU-4 from 26.97 to 50.90. Overall, the results indicate that FL can deliver high-quality conversational AI for public services without centralized data sharing
Read the full abstract ▾
All authors Daniel M. Jimenez-Gutierrez, Albenzio Cirillo, Raffaele Nicolussi, Alessio Beltrame, Andrea Vitaletti
Computation and Language · Cross-listed from cs.LG
FreqLite: A Lightweight Frequency-Decomposed Linear Model with Adaptive Reversible Normalization for Robust Long-Term Time-Series Forecasting
Mirza Samad Ahmed Baig and Syeda Anshrah Gillani
Long-term time-series
forecasting needs models that are accurate yet efficient enough for commodity hardware. Lightweight linear forecasters are remarkably strong in this regime, yet they leave two openings: reversible instance normalization (RevIN) de-normalizes the entire horizon with a single lookback statistic, which is inaccurate under non-stationarity, and time-domain trend/seasonal decomposition relies on a fixed, non-adaptive filter. We present FreqLite, an ultra-lightweight, channel-independent frequency-decomposed linear forecaster: a learnable, lossless, partition-of-unity spectral filter splits the input into bands that are forecast by per-band linear heads and, unlike low-pass-truncation approaches, the high-frequency band is retained and modeled. FreqLite is the best lightweight model on the standard long-term
forecasting benchmarks and, at long lookback (L=336), attains a lower average error than a PatchTST
Transformer (0.3244 vs. 0.3587 MSE) while using 4x fewer parameters, 2.2x less
memory, and 2.2x less time per epoch on a single 4 GB laptop GPU; although modest in magnitude, its improvements are statistically significant under paired Wilcoxon tests across all matched
cells (p < 1e-5). We further introduce Adaptive Reversible Instance Normalization (A-RevIN), a regime-adaptive reversible normalization that strictly generalizes RevIN (recovered exactly when its gate is closed), engages under non-stationarity, and reduces to RevIN without harm on stationary data. We validate this on both a real strongly non-stationary
dataset (ILI, up to ~5% MSE reduction) and a controlled synthetic drift sweep in which A-RevIN's benefit and its learned gate both rise monotonically with injected non-stationarity. Every component is independently ablatable (Linear and RLinear are special cases of FreqLite), and all results are reproducible on commodity hardware.
Read the full abstract ▾
Computation and Language
Benchmarking Local LLMs for Natural-Language-to-SQL Querying in Biopharmaceutical Manufacturing: An Empirical Benchmark on Consumer-Grade Hardware
Sagar Bhetwal, Rajan Bastakoti, Nirajan Acharya et al. (+2 more)
Biopharmaceutical manufacturing organizations operate under regulatory frameworks such as FDA guidance, EU Good Manufacturing Practice (GMP), and the EU AI Act, which can restrict the use of cloud-based artificial intelligence systems. Locally deployed large
language models (LLMs) offer a
privacy-preserving alternative, but their suitability for pharmaceutical manufacturing tasks remains underexplored. This study evaluates four open-source LLMs (Qwen 2.5 Coder 7B, Llama 3.1 8B, Mistral 7B, and Meditron 7B) deployed locally via Ollama for natural-
language-to-SQL generation over a pharmaceutical manufacturing database. A FastAPI-based evaluation platform, PharmaBatchDB AI, was developed using a synthetic Microsoft SQL Server database containing approximately 63,000 records across Batch, Manufacturing Execution System (MES), and Clean-In-Place (CIP) modules. Models were benchmarked on 60 domain-specific natural-
language questions using metrics including SQL extraction rate, SQL compliance, factual consistency, ROUGE-L,
hallucination rate, throughput, and latency. Qwen 2.5 Coder 7B, Llama 3.1 8B, and Mistral 7B generated SQL for all evaluation tasks, while Meditron 7B failed on nearly all tasks due to context-window limitations and poor SQL generation capability. Llama 3.1 8B achieved the highest SQL compliance, whereas Qwen 2.5 Coder 7B achieved the strongest overall text similarity and factual consistency. Performance differences between the two leading models were not statistically significant. The results show that code-tuned general-purpose LLMs outperform a domain-specific biomedical model on structured query generation for pharmaceutical manufacturing data. Although fully local, GxP-aligned NLQ systems are feasible on consumer hardware, current performance levels still require human oversight and downstream validation for regulated use.
Read the full abstract ▾
All authors Sagar Bhetwal, Rajan Bastakoti, Nirajan Acharya, Gaurav Kumar Gupta, Ambika Baniya Bhandari