Artificial Intelligence · Cross-listed from cs.LG
Shallow Queries, Mature Values: Depth-Asynchronous Self-Speculation for Looped Transformers
Guanghao Li, Zihan Su, Hao Yu et al. (+6 more)
Looped
Transformers reuse a shared block across recurrent depths, making autoregressive decoding expensive because every generated
token requires many sequential recurrent passes. Self-speculative decoders reduce this cost by drafting at an early depth and verifying at full depth, but typically bind draft computation to prefix representations from the same recurrent depth. We find that queries and keys approach their final-depth representations earlier than values, and controlled prefix-channel interventions show that mature values substantially improve shallow draft predictions. Motivated by this asymmetry, we introduce Depth-Asynchronous Self-Speculation (DAS), which decouples the depth of draft computation from the depth of verified-prefix representations it reads. Its Mature-V primitive lets shallow queries retrieve full-depth prefix values without additional recurrent computation. We further develop DAS-Wave, which combines depth-asynchronous prefix reads with carried parallel refinement, progressive block growth, and an independent full-depth verifier. Across four recurrent-model checkpoints and mathematics and code workloads, DAS-Wave achieves 4.00–6.96$\times$ mean throughput speedup over paired full-depth autoregressive decoding in the same inference stack. These results identify prefix-information depth as an effective design axis for recurrent self-speculation.
Read the full abstract ▾
All authors Guanghao Li, Zihan Su, Hao Yu, Jinyang Jiang, Tao Ren, Zehao Li, Feng Lu, Ming Tang, Chun Yuan
Artificial Intelligence
The Marathon of Scientific Reasoning: Robustness of Scientific Agents to Perturbations in Multi-Turn Interactions
Xiaoting Lyu, Xinbo Ma, Yufei Han et al. (+5 more)
Large
language model (LLM)-based scientific
agents are increasingly used for scientific problem solving, yet their robustness to imperfections arising during multi-turn interactions remains poorly understood. We introduce SciARP (Scientific
Agent Robustness to Perturbations), a
benchmark for evaluating scientific
agents under scientifically plausible perturbations throughout multi-turn problem solving. SciARP transforms 620 scientific problems into interdependent tasks of 3–13 turns and defines 13 perturbation types spanning problem understanding, evidence processing,
reasoning, and conclusion formation. Clean and perturbed versions of each task are independently executed under matched settings, producing paired live trajectories for evaluating both task success and process reliability. Experiments across eight LLMs from four model families reveal three key robustness characteristics. First, different classes of scientific perturbations exhibit distinct robustness profiles and can decouple task progression from scientific reliability:
agents may continue advancing through the task even after their information or
reasoning has become unreliable. Second, stronger clean-task performance does not necessarily translate into stronger robustness, as models with higher clean-task accuracy can exhibit larger degradation under perturbation. Third, perturbation effects exhibit strong temporal dynamics: they may remain latent for multiple turns before emerging and subsequently propagate through downstream dependencies. Together, these findings show that current scientific
agents remain insufficiently robust to scientifically plausible perturbations, with failures often remaining undetected, propagating, and resisting recovery.
Read the full abstract ▾
All authors Xiaoting Lyu, Xinbo Ma, Yufei Han, Hangwei Qian, Ziyang Lin, Bin Wang, Bin Wang, Wei Wang
Artificial Intelligence
PairPref: When Should Memory Guide the Answer? A Benchmark for Contextual Preference Use
Mingfei Lu, Mengjia Wu and Yi Zhang
Memory-augmented assistants use retrieved preferences to guide their responses. A small change in the situation can change whether a preference is appropriate while barely affecting its
retrieval similarity.
Memory benchmarks typically test whether systems store and retrieve preferences, with less
attention to when those preferences should apply. We introduce PairPref, a
benchmark of contextual preference use. Each pair changes only the situation, keeping the preference, request, and four candidate replies fixed. The preference remains valid in both situations. In the selection track, models must choose the reply that applies the preference only where appropriate. In the free-generation track, they must decide when to apply it without seeing candidate replies. Both tracks use the same 1,227 pairs across 45 preferences and eight situation categories. We evaluate eight models, most of which achieve selection scores ($\Delta$) of 51 to 65 points. In free generation, however, both responses are appropriate for their respective situations in only 3.6% to 18.3% of pairs. Models continue to apply the preference in both situations even with fewer retrieved
memories, alternative presentation formats, and a stricter
prompt. These results show that models still struggle to judge when user preferences apply and respond accordingly.
Read the full abstract ▾
Artificial Intelligence
EOPSA: Efficient On-Policy Self-Distilled Safety Alignment
Qirui Liu, Yichen Sun, Yan Wang et al. (+5 more)
On-Policy Self-
Distillation (OPSD) has emerged as a promising paradigm for safety
alignment, delivering dense,
token-level supervision by distilling from a teacher conditioned on refusal-oriented privileged
prompts. However, we reveal that this paradigm suffers from critical inefficiencies that degrade both training efficiency and general
reasoning capabilities. Specifically, we diagnose two fundamental bottlenecks: (1) supervisory collapse over extended rollouts, where the teacher's corrective efficacy degrades precipitously as the student's generation prefix lengthens, injecting noisy gradients into late-stage
tokens; and (2) gradient dilution from stylistic shifts, where the
distillation objective is dominated by safety-irrelevant stylistic discrepancies induced by privileged prompting, washing out genuine safety signals and impairing base
reasoning. To resolve these issues, we propose Efficient On-Policy Self-Distilled Safety
Alignment (EOPSA), which concentrates computational and gradient budgets exclusively on reliably supervised, safety-critical
tokens. EOPSA incorporates two coordinated mechanisms: (i) Adaptive Rollout Scheduling, which dynamically bounds the generation horizon guided by a novel Teacher Rescue Rate (TRR) metric to operate strictly within reliable supervision regimes; and (ii) Selective
Distillation, which filters out safety-neutral
tokens to restrict gradient updates exclusively to safety-pivotal transitions. Extensive evaluations across
reasoning models up to 32B parameters demonstrate that EOPSA slashes rollout computation by $\sim$50% and backpropagates through merely $\sim$2% of
tokens, consistently outperforming full-
token distillation baselines in both safety compliance and
reasoning retention.
Read the full abstract ▾
All authors Qirui Liu, Yichen Sun, Yan Wang, Yu Mi, Wei Cao, Yue Shen, Zhixuan Chu, Kui Ren
Artificial Intelligence · Cross-listed from cs.CR
SEAD: A State-Based Perspective on Attack and Defense in Tool-Using Agents
Xinjie Shen, Junran Wang, Rongzhe Wei et al. (+1 more)
Language-model
agents increasingly use tools to act on external systems. Earlier actions can alter files, permissions, database records, or other state, making a later routine-looking action harmful. Yet the visible interaction may not reveal the underlying state needed to assess that action. We formulate attack and defense as partially observed state control in SEAD, deriving their design requirements from this shared execution process. Because attackers supply instructions while the target chooses concrete actions, DART decomposes harmful goals into locally plausible steps and uses feedback from actual tool execution to guide trajectory search. The defender must decide before execution with incomplete state evidence. SAGE can therefore investigate relevant state through read-only queries before allowing or blocking each action, including those proposed after a block. We construct an environment-verifiable
dataset integrating controlled initial states, replayable tool environments, and task-specific executable checks. Across four target models, DART improves semantic attack success by 18.8–35.9 percentage points over the competing baseline, with consistent gains under executable verification. On recorded trajectories, SAGE preserves 95.79% of benign trajectories while intercepting 92.73% of harmful paths by the harm-enabling boundary. In online attack-defense evaluation, it reduces DART's executable attack success from 48.0% to 4.0%. SAGE remains effective across four attack methods and generalizes to out-of-domain environments. Our code and data is available at https://github.com/EverywhereSafety/SEAD.
Read the full abstract ▾
All authors Xinjie Shen, Junran Wang, Rongzhe Wei, Pan Li
Artificial Intelligence
Can AI Make Money in Crypto? Measuring the Gap from Backtests to Real Markets
Xingtong Yu, Jiarun Zhou, Guanlin Ding et al. (+12 more)
AI-based trading methods have rapidly evolved from machine learning and reinforcement learning to large
language models (LLMs) and trading
agents, yet their performance is still predominantly assessed through historical backtesting. Such evaluations provide limited evidence of whether a method can generalize to unseen future
markets or whether its backtested performance can be sustained in realistic trading frictions (e.g., latency, slippage, liquidity constraints, and
market impact). We present a unified
benchmark that evaluates representative machine learning, reinforcement learning, LLM-based, and
agent-based trading methods in cryptocurrency
markets through three progressively more realistic stages: historical backtesting, prospective exchange-based paper trading, and real-money live trading. These stages jointly increase temporal realism by moving from historical to unseen future
markets, and execution realism by moving from offline simulation toward live trading. This protocol enables us to quantify the backtest-to-realization gap, identify when performance begins to deteriorate, and compare how this gap differs across major classes of AI trading methods. We further provide a unified open-source system supporting all three evaluation stages, together with a public platform that continuously updates
benchmark results. Code is available at https://github.com/Starlien95/Awesome-TradingAI.
Read the full abstract ▾
All authors Xingtong Yu, Jiarun Zhou, Guanlin Ding, Wenkang Wei, Jiarui Liu, Chang Zhou, Fangzhou Ge, Chenyi Xu, Xikun Zhang, Renqiang Luo, Jie Zhang, Hong Cheng, Xinming Zhang, Hui Zhang, Yuan Fang
Artificial Intelligence
Does Model Uncertainty Track Human Ambiguity? Evidence from Multi-Annotator Vision Benchmarks
Manya Singh and Arjun Pakrashi
Human-model
alignment is critical for trustworthy AI-assisted decision-making systems. Yet, most work evaluates model predictions against single ground-truth labels, overlooking that humans themselves often disagree on labels, a signal of genuine ambiguity. We investigate whether models struggle on the same instances that humans find difficult. We measure this on two
vision datasets (FER+ and CIFAR-10H) where multiple human annotations per image capture human disagreement patterns. We evaluate eight pretrained models across three architectures (ResNet, EfficientNet, MobileNetV3) in two parts: first, whether model uncertainty (softmax confidence,
entropy) correlates with human disagreement, and second, whether predictive multiplicity measures (inter-model disagreement, Jensen-Shannon divergence) do. We find that it does not:
alignment is weak in both dimensions. At the discrete label level, 50.4% of CIFAR-10H images and 33.5% of FER+ images receive multiple valid classifications from humans, while the models converge on only one. These instances represent a critical failure case where humans perceive ambiguity and would request expert review, yet models decide confidently. At the continuous score level, single-model uncertainty correlates weakly with human disagreement ($\rho = 0.24--0.55$), and predictive multiplicity provides only modest improvement. Widely-used uncertainty quantification methods do not reliably identify instances humans find ambiguous. Model uncertainty should not be treated as a trustworthy signal by default for decision-making in high-stakes scenarios.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Scalable GNN-based Knowledge Graph Representation Learning with Efficient Message Passing
Huu Tan Mai, Cuong Xuan Chu, Heiko Paulheim et al. (+1 more)
Graph neural networks (GNNs) excel at representation learning on Knowledge Graphs (KGs), achieving stateof-the-art performance on tasks like link prediction or entity classification. However, their high computational complexity, inherent to their user-defined message passing (MP)
algorithm, still prohibits their widespread adoption, especially for large KGs. Current efforts to mitigate the scalability bottlenecks of GNNs on KGs, such as subgraph sampling, are often task- and model-specific, and do not reliably guarantee lossless (if applicable) runtime/space reductions. To address this, we extend Relational Sparse
Matrix Multiplication (RSPMM), originally designed to losslessly lower the space complexity of composition-based MP with pointwise composition functions, to support more expressive functions (e.g., 2x2 block-diagonal
matrix multiplication, Givens rotation, circular correlation). Our method delivers significant task-independent reductions in runtime and space for current GNNs on KGs and facilitates efficient re-implementations of GNNs that maintain near state-of-the-art performance on challenging KG tasks, for a fraction of computational costs.
Read the full abstract ▾
All authors Huu Tan Mai, Cuong Xuan Chu, Heiko Paulheim, Daria Stepanova
Artificial Intelligence
Xijing Wang, Yinsheng Yao, Jinru Ding et al. (+5 more)
Large
language model (LLM)
agents offer new opportunities for automated analysis in industry. However, rigorous evaluation of such
agents-for example, within power system scenarios-remains hindered: real operational data are confidential, and existing public resources fail to fully capture the chained dependencies and heterogeneous evidence. To address this gap, we propose PowerBench, comprising (1) a generation framework that derives interconnected heterogeneous operational data through a common dependency chain, and (2) a synthetic
dataset generated by this framework. The
dataset covers 761 devices across 100 device types, with 13.35 million hourly telemetry records spanning two years and 24,939 operational documents. Building on this
dataset, we construct 300 questions across three task families that evaluate frontier LLMs' ability to complete analysis tasks that require autonomous evidence
retrieval and
reasoning across interconnected and heterogeneous data under restricted tool calls and time budgets. Results demonstrate that the evaluated frontier LLMs remain challenged on these tasks: the best model reaches only 74.2% joint accuracy. Our trace analysis further reveals that model performance varies across evidence discovery, content
retrieval, tool use,
reasoning over evidence, and answer submission. These findings provide detailed insights for evaluating LLM
agents and guiding their reliable deployment in industry. The framework,
dataset, and
benchmark tasks are available at https://github.com/open-compass/PowerBench.
Read the full abstract ▾
All authors Xijing Wang, Yinsheng Yao, Jinru Ding, Yidong Jiang, Ziwen Xu, Yiwen Jiang, Jie Xu, Dawei Cheng
Artificial Intelligence · Cross-listed from cs.LG
FlexLoop: Depth-Elastic Looped Policies for Adaptive Test-Time Computation in Deep RL
Xun Wang, Ruishuo Chen, Yu Chen et al. (+2 more)
Looped architectures scale computation by reusing the same parameters across recurrent steps, and recent work shows that they substantially improve deep reinforcement learning policies on long-horizon tasks. Since recurrent depth directly controls computation, one may expect looped policies to naturally support elastic inference across recurrent depths. Surprisingly, we find that pretrained looped policies exhibit severe recurrent-depth specialization: reliable decisions are concentrated near the full trained depth, tying deployment computation to this depth even when less computation may suffice. Achieving depth elasticity, i.e., reliable decisions across recurrent depths with adaptive computation at deployment, therefore remains a key challenge. To address this, we propose FlexLoop, a novel post-training framework that converts pretrained fixed-depth looped policies into depth-elastic policies. FlexLoop keeps training on the original RL objective to preserve full-depth capability while performing adjacent-depth policy
distillation to progressively transfer decision quality from deeper to shallower recurrent steps. The resulting policy supports reliable inference across recurrent depths and enables state-wise adaptive inference through recurrent-depth consistency. Experiments on $30$ online and offline long-horizon goal-conditioned environments show that FlexLoop preserves full-depth performance while making shallower depths effective. Keeping competitive performance, FlexLoop reduces average recurrent depth by up to $\bf{43\%}$ and achieves up to $\bf{1.34\times}$ wall-clock speedup in a stress test.
Read the full abstract ▾
All authors Xun Wang, Ruishuo Chen, Yu Chen, Zhuoran Li, Longbo Huang
Artificial Intelligence · Cross-listed from cs.CV
SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis
Hangyul Yoon, Hyungyung Lee, Edward Choi et al. (+1 more)
Vision-
language (VL) pretraining using paired chest X-ray (CXR) images and radiology reports has shown strong potential for medical image understanding. However, existing methods often remain dependent on task-specific finetuning because radiology reports are lengthy, clinically dense, and difficult to align with simple zero-shot
prompts. Recent sentence-level approaches partially address this limitation using clinical phrases extracted by large
language models (LLMs), but they largely overlook the intrinsic characteristics of radiology discourse. In particular, limited positive-pair diversity constrains further gains, while clinically equivalent sentences frequently recur across patients, creating false negatives in contrastive learning. To address these issues, we propose SentZero, an enhanced sentence-centric VL pretraining framework for zero-shot, multi-task CXR analysis. SentZero introduces LLM-based abstract-level sentence structuring and mapping to expand positive-pair diversity, together with an additional loss term to mitigate false negatives. We further introduce sentence-conditioned residual modulation of visual
embeddings, enabling visual features to adapt to the semantic characteristics of each input sentence. Across diverse downstream tasks and
datasets, SentZero improves zero-shot generalization and outperforms prior multi-task zero-shot methods.
Read the full abstract ▾
All authors Hangyul Yoon, Hyungyung Lee, Edward Choi, Eunho Yang
Artificial Intelligence · Cross-listed from cs.LG
Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases
Jiangtao Lin, Bangyang Wei, Yihang Ding et al. (+2 more)
Task-specific fine-tuning can rewrite a
language model's answers beyond the training task, complicating updates that must preserve existing behavior. We introduce ATLAS, which turns retained-domain representations into an input-dependent rule for task adaptation. An activation atlas supplies local reference centers and directional filters to a shared low-rank residual. Target supervision learns the residual, while retained geometry shapes its action throughout training and inference. On Qwen3-8B, ATLAS achieves lower mean retained-output Kullback-Leibler (KL) divergence than all seven published baselines at shared coding-performance requirements, with consistent advantages across multiple training seeds. Structural comparisons identify the contributions of retained reference states and directional conditioning, and answer-level analyses show fewer rewritten mathematical answers and more stable commonsense choices. Experiments spanning five backbones and two retained domains further demonstrate coding gains with reduced retained-output movement. With compact storage and modest decoding overhead, ATLAS provides a practical mechanism for acquiring specialized skills while maintaining continuity in existing responses.
Read the full abstract ▾
All authors Jiangtao Lin, Bangyang Wei, Yihang Ding, Siyi Liu, Yuhan Dong
Artificial Intelligence · Cross-listed from cs.LG
Causal Routing for Unlearning
Bardh Prenkaj, Andrea D'Angelo, Davide Mottin et al. (+4 more)
LLMs cannot forget the way we delete a file. Strangely, we are asked to remove something that was never put anywhere in particular. What the model took from a piece of text is now smeared across billions of weights. Existing methods rewrite all of them to change one thing, and none of them say which part produced that change. To address this, we introduce Causal Routing for Unlearning (CRU) by asking where the concept is expressed in the model and suppressing only that part. One untrained forward pass over the forget set ranks
neurons by how their activations vary. Then, small routing modules on those
neurons gate and suppress only the concepts that need to be forgotten. In CRU, the base model is frozen, and any change in behavior is caused only by the gated
neurons; hence, why the routing is causal. Due to our parameter efficiency (only ~0.01% as many parameters as the base model), unlearning a concept costs 14 GiB, whereas the baselines require 71 GiB. On TOFU, CRU is indistinguishable from the retained model (p > 0.05, KS test) and is never Pareto-dominated, whereas every compared baseline matches its forgetting on the larger-forget batches only by collapsing utility. On RWKU, it achieves an
adversarial-probe recall of 0.052, compared to 0.250 for the strongest baseline, meaning the knowledge is gone, not merely harder to reach. Thus, deciding on the intervention at query time, rather than fixing it beforehand, is the axis along which we argue that unlearning should proceed.
Read the full abstract ▾
All authors Bardh Prenkaj, Andrea D'Angelo, Davide Mottin, Federico Fontana, Davide Gabrielli, Paola Velardi, Stefano Faralli
Artificial Intelligence · Cross-listed from cs.CV
VL-AcneSeg: A Vision-Language Framework for Region-Aware Acne Lesion Segmentation
Sukju Oh, Soo Ick Cho, Dae Hun Suh et al. (+1 more)
Acne assessment is crucial for clinical decision-making, yet traditional grading and counting are subjective and fail to account for lesion size. While area-based assessment has emerged as a promising alternative, acne
segmentation has continued to rely on general-purpose architectures. To address this gap, we propose VL-AcneSeg, a
multimodal framework for acne lesion
segmentation that leverages CLIP and region-level text
prompts to incorporate spatial priors, enabling lesions to be localized across the whole face. Because region-level
prompts indicate which facial areas contain lesions, we report a single global
prompt, which requires no such information, as our primary setting. On our internal clinical
dataset, VL-AcneSeg achieves a Dice score of 0.5082 and an IoU of 0.3407 under this protocol, the highest among all compared methods, including recent
vision-
language segmentation methods that are themselves given region-level
prompts; region-level prompting raises these to 0.5296 and 0.3602. Moreover, lesion area measurements derived from our
segmentation correlate with IGA scores at a level comparable to expert annotations (Pearson r = 0.719 versus 0.658). Notably, our framework maintains consistent performance across external validation
datasets, performing reliably even on uncontrolled smartphone images without requiring additional training or fine-tuning. By pairing a protocol that requires no lesion-location information with area-based severity estimation, this work provides a foundation for objective acne assessment outside the clinic. Our implementation is publicly available at: https://github.com/sukjuoh/VL-AcneSeg
Read the full abstract ▾
All authors Sukju Oh, Soo Ick Cho, Dae Hun Suh, Sukkyu Sun
Artificial Intelligence · Cross-listed from cs.CV
Precise Editing and Flexible Referencing for Interactable Worlds
Xinyao Liao, Xianfang Zeng, Zhu Liang et al. (+5 more)
We present EditWorld, a
video world model for precise editing and flexible referencing in interactable worlds. Existing
video world models primarily focus on
navigation, letting users explore generated worlds but offering limited control over how existing world content is modified. EditWorld extends world modeling from exploration to precise modification by streaming editing instructions and reference images during autoregressive generation. To support these capabilities, EditWorld introduces Gated Causal
Attention for temporally varying editing conditions and reference images, together with a Sparse Context mechanism that maintains a bounded historical context for long-horizon inference. We further adopt joint autoregressive and bidirectional training with annealed self-resampling, and construct a dedicated data synthesis and annotation pipeline that provides supervision for world editing. We also present WBench-Editing to systematically evaluate streaming world editing capabilities. EditWorld achieves the best overall performance on WBench-Editing with an overall score of 73.8 and an editing score of 80.0, substantially outperforming existing methods on editing-related metrics. https://github.com/leoisufa/EditWorld
Read the full abstract ▾
All authors Xinyao Liao, Xianfang Zeng, Zhu Liang, Zhoujie Fu, Qianxun Xu, Jiachi Liu, Gang Yu, Guosheng Lin
Artificial Intelligence · Cross-listed from cs.CR
CoDeL: Co-Evolutionary Defense against Indirect Prompt Injection in LLM-based Agents
Xiao Yang, Yangchen Ou, Yuhan Gao et al. (+3 more)
Large
language model (LLM)-based
agents increasingly rely on external tools and content, exposing them to indirect
prompt injection (IPI). This threat has motivated a wide range of defenses, among which training-based defenses are often regarded as most reliable. However, existing training-based defenses are typically optimized on a static distribution of explicit injections. They learn surface-form cues rather than the boundary between serving the user and obeying an injected objective, and therefore fail when malicious intent is folded into a plausible workflow and deferred for several turns. We present CoDeL, a defense that hardens
agent against an attack distribution it reshapes as it trains. The defender is updated each round via LoRA-based GDPO under a decoupled reward over safety, task progress, and format compliance, so refusing injections and completing the user's task jointly define fitness. To keep supplying it with the failures worth learning from, a co-evolving prober searches over injection rounds, attack methods, and payloads for injections that still penetrate the current defender, guided jointly by attack success and attack latency so that it preferentially mines breaches the defender notices too late. Each defender update invalidates part of the attack population and forces the next round onto a new frontier, turning the defender's own failures into a moving curriculum. Extensive experiments on three IPI
benchmarks, nine baselines, and two base models show that CoDeL reduces attack success rate (ASR) by 88.5% and outperforms other baselines largely (+38.0%). Codes are available.
Read the full abstract ▾
All authors Xiao Yang, Yangchen Ou, Yuhan Gao, Le Wang, Zonghao Ying, Aishan Liu
Artificial Intelligence
When Does Structured Knowledge Help Neural Theorem Proving?
Sareh Nabi, Roland Vogl and Marzieh Nabi
Does structured mathematical knowledge help LLMs prove
theorems in Lean 4? If so, for which models, and does the answer vary by problem? Formal libraries such as Mathlib encode 285,000+ verified
theorems with syntactic dependencies, but the semantic layer mathematicians rely on for discovery (analogies, generalizations, cross-domain bridges) remains implicit. We introduce MathAgent, which builds this layer as a knowledge graph, MathKG, and uses it to augment LLM
theorem provers. MathKG connects 364 Mathlib
theorems and definitions by 9,434 typed semantic edges inferred via LLM-based relation extraction anchored to verified Mathlib declarations. We run a controlled ablation across four augmentation modes (no context, knowledge-graph context, Mathlib
retrieval, both) and five models: Qwen3-8B/32B, their Lean-specialized derivatives Goedel-Prover-V2-8B/32B, and Claude Sonnet 4.6, on miniF2F, plus PutnamBench and MathOlympiadBench for Sonnet. Three findings emerge. (i) Specialization dominates augmentation: Lean fine-tuning adds 33-38 percentage points of solve rate in every mode, and a specialized 8B model beats a $4\times$ larger general one by 29-35 points, while no augmentation mode improves solve rate by more than 3 points. (ii) Augmentation is capability-conditioned: knowledge-graph context helps small models but hurts large ones, with the specialized model gaining more relative to its general base at every scale. (iii) Yet the augmentation modes solve different problems: an oracle selecting the best mode per problem solves 6% to 58% more than the unaugmented prover, a complementarity effect that strengthens on harder problems (32% more on PutnamBench). These results motivate adaptive strategies that select augmentation by model capability and problem. Code, data, and artifacts are available at https://github.com/sarehnabi/mathagent
Read the full abstract ▾
Artificial Intelligence
Escaping Local Views: Discovering Latent Concepts for Interpretable Multi-Agent Reinforcement Learning
Yijie Sun, Sanquan Sun, Yanda Zhu et al. (+3 more)
Efficient cooperation is challenging due to the usual partial observability of each
agent in multi-
agent reinforcement learning. Recurrent networks encode local interaction histories, but their hidden representations provide limited insight into the information underlying individual decisions. To address these challenges, we propose a novel interpretable framework, called escaping local views (ELV), which introduces semantically structured latent concepts to render policy decisions transparent. Specifically, each
agent extracts low-dimensional semantic concepts from its local observation and action-observation trajectory. These concepts are jointly encoded into a contextual latent variable via a variational autoencoder (VAE), which builds a bridge between local views and global semantics. To explicitly model the decision of each
agent, we employ a dual-path
attention mechanism in which one module estimates the salience of individual concepts relative to the global context, while the other captures higher-order cooperative patterns with pairwise concept interactions. Furthermore, we incorporate a concept prediction module that derives an intrinsic reward from next-concept prediction errors, which incentivizes
agents to explore regions of semantic novelty. Experiments in multiple environments verify that ELV not only achieves competitive performance but also explicitly provides how
agents reason about their decisions.
Read the full abstract ▾
All authors Yijie Sun, Sanquan Sun, Yanda Zhu, Yuanyang Zhu, Yaohua Hu, Chunlin Chen
Artificial Intelligence · Cross-listed from stat.ML
Understanding Generalization Requires Universal Induction
Aram Ebtekar, Marcus Hutter and Danica J. Sutherland
Classical statistical theory is insufficient to explain the successes of general-purpose AI models, because it depends on handcrafted inductive biases that it cannot justify. No Free Lunch (NFL)
theorems force any learner that beats chance on some environments to underperform on others. We might hope that past experience informs which environments to expect, but NFL applies equally to meta-learning. Thus, any method that makes meaningful predictions necessarily begins with an inductive bias external to the data. Choosing to bias toward short programs yields Solomonoff induction (SI), whose performance is competitive against all computable learners - albeit up to "constants" that become large when comparing against specialized methods that exploit background information. We therefore relativize SI to an information vantage point, biasing toward short programs with access to all preexisting information. This reframes the inductive bias: instead of seeking some absolute notion of simplicity, we favor accessibility with respect to our vantage point. An
algorithm can only outpredict the relativized SI to the extent that its code contains additional information about the data, and no
algorithm can generate such information. While SI is incomputable and hence not a practical
algorithm, it provides a formal optimum for inference in the limit of infinite compute, and there is evidence to suggest that frontier AI systems roughly approximate it. Thus, the only known answer to meta-NFL is rooted in algorithmic information theory, which we should expect to play a fundamental role in explaining the generalization behavior of modern (and future) AI systems.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
SPACE-LoRA: Allocating Activation-Subspace Protection for Continual Learning
Seunghyun Yoo, Kiseok Kim, Hyeontae Joo et al. (+2 more)
This study addresses the catastrophic forgetting problem that occurs when sequentially learning successive tasks using Low-Rank Adaptation (LoRA) from a lifelong learning perspective. While existing approaches have primarily constrained parameter updates or learning subspaces to reduce interference with past knowledge, they have not fully considered additive interference. This occurs when a newly added residual adapter on top of a fixed past model generates non-zero responses along input directions important for old tasks, thereby altering previous predictions. To this end, we propose Subspace Protection with Allocated Capacity for Efficient Continual Adaptation (SPACE-LoRA). SPACE-LoRA directly suppresses the responses of the new residual branch along input activation directions that are important for old tasks and adaptively determines the protection coverage for each module based on past-task sensitivity estimated via a common Fisher sensitivity-based coverage target. Under a fixed LoRA rank, this approach adaptively adjusts module-specific protection coverage while suppressing interference along input directions sensitive to old tasks. We assess the effectiveness of activation-subspace protection in mitigating catastrophic forgetting and examine the role of sensitivity-guided protection in continual learning across diverse tasks. Code is available at https://anonymous.4open.science/r/SPACE-LoRA-7864.
Read the full abstract ▾
All authors Seunghyun Yoo, Kiseok Kim, Hyeontae Joo, Junyeop Bang, Hwangnam Kim
Artificial Intelligence · Cross-listed from cs.CV
Modeling Whole-Slide Images as Dynamic Tumor Microenvironment Fields
Lei Wu, Jiashuai Liu, Di Zhang et al. (+9 more)
Due to the gigapixel-scale nature of whole-slide images (WSIs), weakly supervised WSI analysis is commonly formulated as a multiple instance learning (MIL) problem, where patch-level features are aggregated into slide-level representations. However, diagnostic and prognostic evidence often arises from spatially coherent tumor microenvironment regions and their interactions, rather than isolated patches alone. Existing patch-level or static region-based methods usually overlook how tissue regions should be adaptively formed and subsequently evolved through microenvironment interactions across heterogeneous boundaries. In this paper, we propose Concept-Guided Tumor Microenvironment
Evolution (TMEvolve), a reaction-
diffusion-inspired framework that models WSIs as latent tumor microenvironment fields over discrete patch graphs. TMEvolve instantiates this view as a learnable graph-discretized
evolution process over patch neighborhoods. It first forms adaptive soft tissue regions as coherent microenvironment units, then performs pseudo-time
evolution through two complementary local dynamics: intra-region
diffusion, which stabilizes latent states within coherent tissue compartments, and concept-guided boundary flux, which propagates visual feature signals and
language-derived concept signals across heterogeneous region interfaces. The evolved microenvironment regions are finally aggregated for slide-level prediction. We evaluate TMEvolve on six
datasets across three weakly supervised WSI tasks: survival prediction, gene expression prediction, and histological subtype classification. TMEvolve consistently improves over representative MIL methods, pathology foundation models, and concept-guided baselines. Ablation studies and visualizations further support the effectiveness and interpretability of TMEvolve, highlighting the value of dynamic region modeling and boundary interaction.
Read the full abstract ▾
All authors Lei Wu, Jiashuai Liu, Di Zhang, Zhangpeng Gong, Yingkang Zhan, Yi Niu, Jiusong Ge, Chunze Yang, Kai Yi, Mireia Crispin-Ortuzar, Chen Li, Zeyu Gao
Artificial Intelligence
CORTEX: Learning to Share and Specialize in Dense Language Models
Chuiyang Meng, Ming Tang and Vincent W.S. Wong
Large
language models are trained on heterogeneous data mixtures, where different knowledge domains require both shared knowledge and specialization. Existing modular approaches typically impose explicit components or discover modules through interpretability analysis after training. In this work, we propose CORTEX, a learning dynamics-inspired framework that learns internal modularization within dense
language models. CORTEX partitions trainable
matrices into parameter groups and learns module assignments from domain-conditioned gradient and cross-domain gradient similarity. We introduce the selective lesion score and module-domain mutual information to characterize the target-domain lesion effects and
alignment, and analyze how module assignment affects the trade-off between assignment bias and update magnitude. Experiments with 160M, Qwen3-8B, and Qwen3-32B backbone models show that CORTEX achieves the highest synthetic-domain exact match and largest average perplexity reduction, while remaining competitive on real-domain evaluations and forming identifiable modules.
Read the full abstract ▾
Artificial Intelligence
Social Circuits behind Multi-agent Echo Chambers
Chuiyang Meng, Wenlu Yu, Ming Tang et al. (+1 more)
Language-model
agents exchange messages to combine evidence, but their communication can also create echo chambers that reinforce shared errors. However, overall task performance does not explain how a message changes the receiving
agent's internal activations and affects its decision. In this work, we introduce Social
Circuits, a framework for tracing message effects through receiver activations. We compare the receiver's answers before and after changing a message. Then, we restore selected activations recorded under the original message to determine how much of the message effect these activations reproduce. Based on Social
Circuits, we propose
Circuit-Guided Deliberation (CGD), which learns to select useful messages using receiver activation changes. We establish when activation replacement preserves receiver decisions and bound the gap between CGD's task performance and the best achievable through message selection. Experiments show that receiver activation changes explain the message effects and guide message selection that improves the task performance. Across three models and four
datasets, CGD achieves the highest or joint-highest average accuracy in our main comparisons while generating fewer
tokens than multi-
agent baselines.
Read the full abstract ▾
All authors Chuiyang Meng, Wenlu Yu, Ming Tang, Cheng Li
Artificial Intelligence · Cross-listed from cs.CL
Wanqi Zhou, Jiawei Lu, Yang Wang et al. (+4 more)
Long-term
memory is essential for
language agents to maintain coherent and effective behavior over extended, multi-session interactions. Existing
memory systems mainly use
retrieval at read time, while write-time
memory formation still relies on direct extraction or compression. However, when future information needs are unknown, compressing an entire interaction in one pass can overlook locally important details that may matter later. To this end, we introduce RIME, a
retrieval-induced
memory framework that shifts
memory construction from monolithic compression toward evidence-centered integration. RIME uses generic self-questions to retrieve focused dialogue evidence and grounds
memory formation in both the retrieved evidence and relevant historical
memories, which are jointly reconciled into an evolving
memory bank with temporal and provenance information. At inference time, compressed
memory serves as the primary rather than the sole source of evidence: when it cannot support an answer, RIME retrieves relevant source dialogue together with its local context to recover information omitted during
memory formation, without resorting to full-history processing. Extensive experiments on LoCoMo with Qwen3-235B-A22B and GPT-5.6 Sol show that RIME consistently achieves the best performance across all three quality metrics among the compared methods, while requiring substantially fewer query-time LLM
tokens.
Read the full abstract ▾
All authors Wanqi Zhou, Jiawei Lu, Yang Wang, Zhaolong Xing, Zhen Chen, Ai Han, Haoyue Shi
Artificial Intelligence · Cross-listed from cs.CL
AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering
Chanhee Park, Jeongho Yoon, Sungbin Han et al. (+2 more)
Agentic tasks require a large
language model to interact with the world, navigating information and gathering evidence across multiple steps with restricted resources. Due to this complexity, agentic task failures arise from various sources, and pinpointing these failure causes is essential to diagnose and improve agentic systems. Existing
benchmarks, however, tend to focus on a single leaderboard score, leaving the underlying failure modes opaque. To fill this gap, we introduce AgentHop, a diagnostic
benchmark of 1,011 multiple-choice questions paired with a controlled seven-tool sandbox under fixed
token, turn, and tool-call constraints. AgentHop reveals model vulnerabilities by dissecting a single accuracy score along four axes of
agent operation:
retrieval, synthesis, tool-call, and resource management. Across 19 models, we find that behavior clusters by model family, with tool-call signatures revealing distinct family fingerprints: GPT models commit early, Anthropic and GLM checkpoints verify before committing, DeepSeek and Kimi over-search, and Gemini-3 Pro stays balanced. Decomposed axes further expose within-family structure: Claude Opus 4.6 and Sonnet 4.6 land within one accuracy point yet diverge on
retrieval-versus-synthesis emphasis, with Opus retrieving more and Sonnet synthesizing better. We release the full
benchmark set and the harness to support diagnostic
agent benchmarking.
Read the full abstract ▾
All authors Chanhee Park, Jeongho Yoon, Sungbin Han, Hyeonseok Moon, Heuiseok Lim
Artificial Intelligence · Cross-listed from cs.CL
Zero-Shot Cue-Grounded Topic Segmentation of Spoken Documents
Suhwan Choi, Myeongho Jeon and Myungjoo Kang
Topic
segmentation structures spoken documents into coherent sections, facilitating
navigation and downstream understanding. The appropriate granularity can vary substantially, ranging from broad thematic shifts to fine-grained subtopics. Existing LLM-based segmenters, however, often struggle to adapt to this variation, causing them to either merge distinct subtopics or over-segment coherent themes. To address this, we introduce Cue-Grounded
Segmentation (CGS), a training-free framework that operates without any task-specific supervision. CGS first identifies phrases that explicitly signal the start of a new topic and uses their sentence positions as segment boundaries. When such cues are insufficient, it falls back to semantic
segmentation, guided by the document structure inferred during cue extraction. Across six
benchmarks and six LLM backbones, CGS consistently outperforms existing baselines, remains robust to noisy ASR transcripts, and achieves these gains with low API cost on proprietary models.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Coding Agent Memory Post-training: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via Reinforcement Learning
Lirui Luo, Kelong Mao, Heming Xia et al. (+10 more)
Language-model
agents increasingly tackle long-horizon tasks whose interaction histories exceed the model's active context. Recent work has begun to use reinforcement learning to make
memory control part of the policy, often relying on predefined
memory tools within domain-specific training environments of relatively short horizons. This setup ties learned
memory behavior to environment-specific interfaces that lie outside the base model's pre-training and must be learned from scratch, so even after post-training,
agents struggle to use
memory in long-horizon tasks. To address these limitations, we introduce Coding
Agent Memory Gym (CAMG), a suite of long-horizon agentic-RL environments spanning Shop, Coding, DeepResearch, and AutoResearch. Alongside each environment's native task interface, CAMG provides executable shell access and an episode-persistent workspace, enabling
agents to create, revise, search, and reuse files as
memory throughout an episode. We also introduce CAMG-RL, which trains a single policy jointly across all four environments with fully asynchronous PPO, learning this file-based
memory behavior directly from downstream task reward, and we train CAMG-RL-4B and CAMG-RL-9B from Qwen3.5 models of matching size. On SWE-bench Verified and MLE-bench Lite, CAMG-RL-4B and CAMG-RL-9B are competitive with Qwen3.5-35B-A3B and Qwen3.5-122B-A10B, respectively.
Read the full abstract ▾
All authors Lirui Luo, Kelong Mao, Heming Xia, Rongqing Li, Xinwei Yang, Luyu Chen, Kieran Wong, Yudong Guo, Xinrui Wang, Jiayin Zhu, Simiu Gu, Sulong Xu, Cong Fang
Artificial Intelligence
Beyond End-to-End Black Box Mapping: An Intentional Agent Framework for Cognitive-driven Facial Reaction Generation
Hanzhong Zhang, Jindong Wang and Siyang Song
Automatic human-like facial reaction generation (FRG) is essential for building intelligent systems that can engage in human-computer interaction (HCI). While diverse and context-appropriate facial reactions can reflect latent appraisal and affective processes in human interaction, most existing FRG methods rely on end-to-end architectures that directly map speaker behaviours to listener expressions without an explicit intermediate internal state. We reformulate FRG as generation mediated by a structured internal-state process and propose the Intentional
Agent, which shifts FRG from direct stimulus-response mapping to stimulus-grounded generation through explicit intermediate states. To represent temporal internal-state
evolution, we propose an internal dynamics model that integrates emotional drives with an iterative Inner Thought Flow (ITF) within a structured intermediate state used for subsequent generation. This state can continue to update during conversational silences. Furthermore, to bridge abstract internal states with physiological actions, we formulate FRG as a downstream affective mapping from this latent thought flow to facial expressions. Experiments on the REACT 2025
dataset show an FRDist of 72.39 and an FRDiv of 0.5057; perceptual plausibility is evaluated separately through blinded human ratings. A blinded human evaluation of 96 reactions found no significant difference in mean score between Full and ground truth ($5.527$ vs.\ $5.195$, $p_{\mathrm{Holm}}=.076$), while Full significantly outperformed Event-Triggered and Heuristic-Only (both $p_{\mathrm{Holm}}<.001$). The Reaction Quality Scorer (RQS) correlated strongly with human judgements (Pearson $r=.855$; Spearman $\rho=.821$, both $p<.05$), supporting its use as an automatic metric. These results underscore the immense potential of endogenous dynamics in building highly autonomous, human-like
agents.
Read the full abstract ▾
Artificial Intelligence
OSPD: On-Policy Self-Distillation for Persona-Consistent Dialogue
Rui Xu, Yikai Zhang, Aili Chen et al. (+3 more)
Maintaining persona consistency across multi-turn dialogues remains a core challenge for role-playing
language models. Off-policy
distillation from external teachers incurs distribution mismatch that compounds across dialogue turns, while reinforcement learning struggles with reward ambiguity inherent in subjective persona fidelity. We propose OSPD, an on-policy self-
distillation framework where the same model serves as both teacher and student under asymmetric information: the teacher receives a complete character profile while the student sees only a brief summary, and the student generates trajectories from its own policy. We find that teacher confidence in role-playing dialogue exhibits a bimodal structure—sharply peaked at character-critical
tokens yet diffuse at generic utterances—and introduce role-aware divergence switching to match this structure. A progressive trait masking curriculum further forces staged internalization of character knowledge along semantic dimensions. Experiments on CharacterBench, CharacterEval, and SocialBench show that OSPD substantially improves persona consistency over supervised fine-tuning and multi-turn RL baselines, without requiring any external teacher or reward model.
Read the full abstract ▾
All authors Rui Xu, Yikai Zhang, Aili Chen, Zicheng Zhao, Xu Yinghui, Libo Wu
Artificial Intelligence
Mathematics for and by human cognition: A resource-rational search for bottlenecks in problem-solving
Sneha Aenugu
Human cognitive constraints are generally viewed as limiting factors in problem-solving. We argue that these constraints can instead play a critical role in driving advances in mathematics and beyond. We propose a theory of mathematical abstraction as a resource-rational search for bottlenecks in problem-solving. Bottlenecks arising from cognitive constraints create pressure to restructure existing knowledge, potentially giving rise to novel formalisms with applications beyond the problems that originally motivated them. Drawing on episodes from the history of mathematics, we illustrate how such bottlenecks can drive the development of novel abstractions and examine how cognitive constraints and affective responses shape this process. Finally, we discuss the implications of this account for machine mathematical discovery and argue that incorporating human-like constraints may facilitate the discovery of useful mathematical abstractions.
Read the full abstract ▾