Artificial Intelligence
Learning response-aware patient dynamics for respiratory support
Xiaolei Lu and Shamim Nemati
Respiratory support can shape the short-term physiological trajectory of critically ill patients, but patients receiving the same intervention may follow different physiological trajectories. Clinical patient dynamics models typically predict future states from recent physiology and recorded interventions, while physiological change is mainly represented through the predicted future state. We propose a response-aware patient dynamics model that explicitly represents physiological change during autoregressive state updating. The model decomposes predicted physiological change into state-dependent baseline dynamics and respiratory-support-associated deviations, with room air providing a reference for the decomposition. We provide a formal analysis of this reference-anchored formulation. A response pathway encodes the predicted physiological change and uses it to update the latent patient state across the forecast horizon. Across ICU cohorts from two independent institutions, the proposed model achieves comparable overall trajectory prediction to patient dynamics baselines, with more consistent improvements when physiological states are changing.
Read the full abstract ▾
Artificial Intelligence
Agentic Network Traffic Monitoring
Manuel Tsoukatos, Hayden Jananthan and Jeremy Kepner
As the use of agentic artificial intelligence increases in nearly every industry, there exists a widening attack surface. It is necessary to monitor
agents to ensure that
agents are acting in a way that is aligned with the users intent.
Auditing an
agent's network traffic provides a clear record of the
agent interactions. This work presents a novel approach to monitoring the network traffic of agentic systems using complex valued hypersparse traffic
matrices by integrating DBOS (DataBase OS), the OneSparse PostgreSQL database, and the GraphBLAS math library. To develop these concepts an agentic simulator was constructed, allowing a varying numbers of AI
agents to collectively survey a virtual environment using different strategies. The resulting network traffic
matrices enable easy monitoring of the AI
agents.
Read the full abstract ▾
Artificial Intelligence
Forecasting Intraday USD/CAD Exchange Rate with News-Derived Monetary-Policy Signals
Maya Kodeih, Aliaa Alnaggar and Mucahit Cevik
Monetary-policy announcements and central-bank communications play a central role in foreign exchange
markets, yet their qualitative, unstructured form makes their
forecasting value difficult to quantify. While prior research has largely focused on sentiment extracted from financial news, comparatively little is known about the relative contribution of different dimensions of monetary-policy communication. Existing studies primarily evaluate whether textual information improves overall
forecasting performance but provide limited insight into which communication channels drive such improvements. To address this gap, this paper introduces a statistical attribution methodology that decomposes monetary-policy communication into interpretable channels and quantifies their incremental
forecasting contribution under false-discovery-rate control. Monetary-policy news is transformed into structured communication signals using large
language models (LLMs) and temporal feature engineering. These signals are evaluated using rolling-window experiments with tree-based machine-learning models. The results show that monetary-policy communication contains measurable predictive information. Attribution analysis shows that predictive value is concentrated in a small subset of signals, with communication timing providing the strongest individual feature-level contribution, targeted communication-activity measures also contributing positively, and LLM-derived sentiment providing complementary information at the group level. The findings indicate that communication-based
forecasting value extends beyond sentiment alone and that attribution, rather than aggregate accuracy alone, is central to evaluating news-derived signals.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.GT
Learnable Randomization as Commitment Against Adaptive Optimizers
Zihan Deng, Chuanzhi Xu, Xiaozhen Zhong et al. (+2 more)
A pricing page can walk the posted price up to the last amount a buyer still accepts, a recommender can hold back a better item for a barely acceptable promoted one, and a classifier can shift its boundary once applicants change their features. The system predicts the response and then picks the menu that serves its own objective, so the surplus above the user's cutoff is taken. Playing the single best action publishes that cutoff, while noise on actions the user would never take throws away payoff and teaches the platform that a worse menu is still acceptable. We study unpredictable near-optimal policies (UNOP), which mix uniformly on near-best actions that remain individually rational. The mixture is a commitment about the response. On a finite price grid, when the best sure-demand price strictly out-earns the randomized band, a seller who already knows the curve posts below the band, and the purchase that occurs is deterministic. Knowing that curve is not the same as predicting the next draw. The mixture can be learned and the optimizer can match its best response, while the user's payoff stays higher because the mixture changes which action is targeted. In pricing and in policy-aware recommendation this leaves more surplus than greedy play when the platform optimizes against the curve and more than one action is acceptable. The gain goes away under quality ranking, a singleton near-optimal set, a wrong utility estimate, or a short-horizon explorer. That is also where mixing should be turned off if the other side is trying to cooperate.
Read the full abstract ▾
All authors Zihan Deng, Chuanzhi Xu, Xiaozhen Zhong, Haoyang Li, Junjie Huang
Artificial Intelligence · Cross-listed from cs.LG
Continual Learning via Self-Probe Gradients
Dongkyu Cho, Rumi Chunara and Sungmin Cha
Adapting pretrained models to new data can cause catastrophic forgetting of previously learned behavior. When only a few past samples remain, they give continual learning methods sparse and narrow evidence about what to preserve. We show that
language models can expand this evidence through self-probing, in which the frozen model generates new inputs from the retained samples and records its own predictions on them. Unlike prior work that replays such data as training examples, our method, CPLUS uses self-probe and past-sample gradients to scale down parameter updates that conflict with prior behavior. Experiments with five
language models on four
benchmarks show three results. First, the same probes preserve more prior behavior as gradient signals than as replay data. Second, CPLUS learns the new data while consistently reducing forgetting more than existing baselines, especially when past data are scarce, and this protection extends to
benchmarks not used for training. Third, we observe that CPLUS also becomes more effective as models grow: within the Qwen3 model family, it recovers an increasing share of the forgetting caused by standard fine-tuning.
Read the full abstract ▾
Artificial Intelligence
Mandela-Bench: Multimodal Models Remember Canonical Images Instead of Seeing Them
Yicheng Bao, Zhenkun Gao, Xiahui Guo et al. (+7 more)
Historical photographs and other canonical images can now be edited seamlessly with a single instruction, often leaving no reliable pixel-level trace. In such cases, the only evidence of manipulation may be a fact about what the image depicts. Existing
benchmarks instead rely on generator artefacts, image-caption inconsistencies, visual implausibilities, or external references, and therefore do not test whether a model can use its own world knowledge to verify a recognized image. We introduce Mandela-Bench, containing 1,507 edits of canonical images: 1,359 knowledge-only forgeries, each contradicting one verifiable fact, and 148 anchor-free controls that preserve the editing process without introducing a factual contradiction, together with 474 untouched originals. We score not only whether a model detects a forgery, but whether its explanation identifies the inserted entity or the fact being violated. Across 36
multimodal models, from 0.8B parameters to frontier scale, we find a consistent failure mode. When a public figure is removed from a familiar photograph, models still name that person in up to 72.7% of responses. Some models can distinguish the replacement face from the original when shown in isolation, yet still judge the full edited photograph as authentic. Providing the true event and date does not improve knowledge-grounded detection, whereas providing the same information after cropping away the recognizable composition does. Even under explicit verification
prompts, only one of the 36 models meets the KGR criterion on at least half of the forged images. These results suggest that the failures cannot be explained by missing knowledge or inadequate perception alone. Instead, they are consistent with recognition biasing verification toward the remembered canonical image rather than the observed edit.
Read the full abstract ▾
All authors Yicheng Bao, Zhenkun Gao, Xiahui Guo, Mingqian Yang, Xueheng Li, Bangwei Liu, Mingang Chen, Lijun Li, Xuhong Wang, Xin Tan
Artificial Intelligence · Cross-listed from cs.LG
The Extender: A Log-Structured Transformer
Jakob Eriksson
We introduce the Extender, a log-structured variant of the standard
Transformer architecture. In a standard
Transformer, each layer communicates with subsequent layers exclusively via the residual $\mathbf{h}$, a superposition channel. The Extender adds a concatenation channel $\mathbf{x}$: each layer $\ell$ emits both a residual update $\delta_\ell$ which is added to $\mathbf{h}$, and a much smaller extension $\epsilon_\ell$ which is appended to $\mathbf{x}$. While both the FFN and $\mathbf{q}$ see $\mathbf{h}$, the
attention $\mathbf{kv}$ projections take only $\mathbf{x}$ as input. As a result, the fully extended $\mathbf{x}$ contains the complete input for the $\mathbf{kv}$ projections of all layers, reducing the persistent
attention memory footprint from $2Ld_{model}$ to $\sum|\epsilon_\ell|$. We find that with $|\epsilon_\ell|=32$, the Extender matches
Transformer accuracy on short-context (CORE) tasks at 199M-924M parameters, and exceeds
Transformer accuracy on long-context (RULER) workloads, again at 924M parameters. For our 1664-wide, 924M model, the Extender's persistent
attention memory footprint is $104\times$ smaller than MHA. The
memory savings grow with model width.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
How Far Do Persona Effects Generalize in Language Models?
Yufan Zhou, Yuxuan Liu, Enze Ma et al. (+6 more)
Persona
prompts ask
language models to answer as particular kinds of people. We test whether relationships learned from these effects predict responses to new questions and remain useful across models and
prompts. Across 57 attributes, three behavioral domains, and seven pairs of open 7 to 9B checkpoints, persona effects can be predictable without being portable. Separate attribute and task gains improve prediction beyond shared scaling significantly in OLMo-3 and Qwen2.5, with the most robust evidence in OLMo-3. In that model, target refitting significantly outperforms gains borrowed from each of the other six pairs. Across model transfers, borrowed gains with one amplitude underperform shared scaling in most directions; allowing two target parameters removes the significant losses but yields no significant benefit over target shared scaling. After rewording, refitting significantly outperforms reuse with one amplitude in all six tested pairs, while changes of examples or country context often preserve reuse value. In the tested
prompt transfers, regularized updates outperform both reuse strategies in median at 64 target questions per attribute. A separate survey comparison finds that selecting the more responsive checkpoint can worsen human fit; responsiveness is confounded with training status, and temperature calibration largely removes this cost but not errors in group ordering. Within the tested gain representation, apparent transfer can come from target calibration; source relationships must add predictive value beyond calibration and regularization. Code and data are available at https://github.com/thzva/persona-gain
Read the full abstract ▾
All authors Yufan Zhou, Yuxuan Liu, Enze Ma, Lyumanshan Ye, Zhongqi Yue, Robin De Croon, Yucheng Jin, Katrien Verbert, Zhao Wang
Artificial Intelligence
Readout is not Recovery: Dissociating Coordinate Emission from Visual-Corruption Repair in Vision-Language Models
Drandreb Earl Juanico
VLM bounding-box localization is both
language generation and spatial commitment. Parseable fields such as bbox_2d make localization easy to score, but dimensions that emit coordinate
tokens need not repair localization after visual evidence is damaged. We study this readout/recovery separation in Qwen3-VL-4B-Instruct on single-object COCO grounding. We compare clean coordinate-
token readout rankings with corruption-derived repair rankings, using object-mask endpoint replacement for recovery and clean-input flooring for depth localization. In Qwen3-VL, coordinate-
token rankings are inert through layer 24, load-bearing from layers 32-35, and peak at layer 34; corruption-derived rankings harm layers 16-24 but become beneficial near layer 35/final. A Kimi-VL-A3B diagnostic shows a matching output-proximal transition despite a different box format. Object-mask recovery separates rank budgets: $k=250$ shows necessity, $k=500$ shows Top-$k$ restoration above random, and $k=d/2$ is largely capacity-driven. Partial-occlusion sweeps reveal that high-overlap coordinate-
token sets can hurt at $k=1000$ and help mainly at half-width, while population corruption-derived sets provide no reliable fixed repair set. Edge-attribution patching shows coordinate-
token paths are high precision but low recall for detection recovery, and RMSNorm quasi-layer controls do not close the endpoint-repair gap. Endpoint coordinate triage is therefore a useful
circuit prior, but occlusion recovery requires a separate
benchmark.
Read the full abstract ▾
Artificial Intelligence
Adaptive Consistency Graph for Long-Horizon Agents
Jiecong Wang, Hao Peng and Zhanyi Wang
Large
language model
agents can often make reasonable local decisions on short tasks, yet their performance degrades when success requires long sequences of dependent actions and tool calls. During execution, task requirements, historical evidence, and the current execution state may gradually become disconnected, so later decisions can drift from the original objective. We study this problem by introducing the Adaptive Consistency Graph (ACG) for long-horizon execution. ACG incrementally organizes execution evidence and its provenance in a persistent graph, then constructs a temporary requirement-centered view for each decision under a bounded context budget. Rather than replacing the base
agent's planner or tool executor, ACG provides a structured and traceable context view for each decision. In the matched evaluation, ACG improves GPT-5.6-luna's average success from 44.5% with ReAct to 50.2%, with the largest gain on BrowseComp-Plus (73.5% versus 62.4%). We further analyze trajectory structure and inference cost to characterize this improvement. Our code is available at https://github.com/yunsaijc/Adaptive-Consistency-Graph.
Read the full abstract ▾
Artificial Intelligence
Action Shaping: Policies Absorb What They Can Express
Yanjun Chen, Jinghan Wang, Xiaoyu Shen et al. (+2 more)
Reward shaping has a
theorem: a potential-based term can be removed without changing the optimal policy. The same practice on the action channel, an offset added in training and dropped at deployment, has no
theorem. Nothing cancels an action offset, so the correction is kept at deployment or removed without a guarantee. We call it action shaping and state its principle. A trainable policy absorbs an offset its own output layer can reproduce exactly, which is what we mean by express; what is absorbed can be removed with the return intact. Its minimal instance is a zero-initialized linear head behind a learnable gate, added to an actor that trains through a learned action-value function, with no penalty or schedule. The gate rises and then falls on its own, for deterministic and
stochastic actors alike, and on 20 tasks removing the head costs almost nothing. The condition is exact reproduction, not capacity: a nonlinear head with more parameters is not absorbed, and in a paired control, one linear path added to a nonlinear base head restores absorption. Exact reproduction gives the loss a flat direction that gradient noise drifts along, and the offset's amplitude indicates, before removal, what dropping the head will cost. Action shaping thus gains the counterpart of the shaping
theorem, a condition for absorption, together with the mechanism behind it and a diagnostic that reads it. Policies absorb what they can express, and only that.
Read the full abstract ▾
All authors Yanjun Chen, Jinghan Wang, Xiaoyu Shen, Wenjie Li, Wei Zhang
Artificial Intelligence
CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning
Xin Yan, Zhengbo Jiao, Jiaqi Liu et al. (+11 more)
Reinforcement learning enables computer-use
agents to improve through interaction with real software environments, including websites and desktop applications. However, conventional deployments replicate an initialized runtime for each independent rollout, even when trajectories use the same software, incurring repeated
memory and initialization costs as the number of parallel environments grows. Does an independent computer-use environment require an independent execution runtime? Our key observation is that trajectories require independent mutable state, while initialized application runtimes can be reused across concurrently evolving environments, making state the natural unit of environment independence. Guided by this observation, we introduce CUA-Sandbox, which separates private state capsules from shared runtimes through state-scoped execution and transactional lifecycle operations, including resets and branches, while retaining the original software interfaces and task evaluators. Experiments show comparable or improved task success relative to Docker, while substantially reducing rollout and resource costs. CUA-Sandbox achieves up to a 6.20x increase in rollout throughput, a 9.2x reduction in per-environment
memory, and a 504x reduction in incremental storage.
Read the full abstract ▾
All authors Xin Yan, Zhengbo Jiao, Jiaqi Liu, Zhenglin Wan, SiYuan Ma, Xuliang Yu, Tianyi Jiang, Chubin Zhang, Pengfei Zhou, Wangbo Zhao, Xingrui Yu, Bo An, Yang You, Ivor Tsang
Artificial Intelligence
Retrospective Distillation Attribution via Normalized Response Similarity
Minwoo Jang, Jaechang Kim, Minhyeon Oh et al. (+2 more)
Model
distillation transfers capabilities through supervised fine-tuning (SFT) on teacher responses, often collected from commercial APIs, raising questions of model provenance. Existing
distillation attribution methods have been largely evaluated on students immediately after the SFT step. However, a distilled model may undergo further SFT, preference optimization, or reinforcement learning before release, while an auditor may lack access to the pre-
distillation checkpoint required by reference-based attribution. To close this gap, we propose SCOUT, an output-only method that aggregates recurring *syntactic patterns* into candidate profiles, filters low-contrast patterns, and calibrates student–candidate distances against inter-candidate distances. SCOUT supports attribution and abstention using only current texts, without model weights,
token likelihoods, or historical checkpoints.
Auditing publicly released descendants of distilled models spanning diverse post-training objectives, SCOUT consistently identifies the
distillation source. Furthermore, tracing teacher-associated *syntactic signatures* along training trajectories reveals that they emerge during
distillation and persist through subsequent preference optimization and reinforcement learning.
Read the full abstract ▾
All authors Minwoo Jang, Jaechang Kim, Minhyeon Oh, Jeongyeon Hwang, Jungseul Ok
Artificial Intelligence · Cross-listed from cs.MA
Self-Evolving Multi-Agent Symbolic Discovery for Financial Fundamental Analysis
Kelvin J.L. Koa, Filip Orestav, Shengqiong Wu et al. (+2 more)
While symbolic regression (SR) has been successfully used in science to discover new equations, its use in financial valuation is hindered by several limitations. Whereas the natural sciences provide objectively correct relationships, financial valuation constitutes a distinct class of symbolic discovery problems, as it admits multiple valid perspectives, operates under non-stationary
market conditions, and involves noisy, continuous performance signals. In this work, we propose Multi-
Agent Fundamental Analysis with Symbolic Adaptive learning (MUFASA), a hierarchical multi-
agent framework for symbolic discovery in finance. MUFASA introduces (1) disentangled equation discovery via specialized
agents representing distinct valuation perspectives, (2) a meta-coordinator that performs hierarchical-level
reasoning over
market context information, and (3) a
memory mechanism that reasons over statistical performance summaries (e.g., accuracy, stability, and tail risk) to guide learning under noisy feedback. Experiments across
datasets from multiple countries show that MUFASA achieves state-of-the-art performance on the valuation task compared to classical finance methods, financial large
language models, and SR approaches, while simultaneously producing interpretable equations, which we share with the community. We also make publicly available the distilled learnings across
evolution iterations and context-dependent strategy weights, which might offer useful insights for future research on financial fundamental analysis.
Read the full abstract ▾
All authors Kelvin J.L. Koa, Filip Orestav, Shengqiong Wu, Michael J. Wooldridge, Ke-Wei Huang
Artificial Intelligence · Cross-listed from cs.CV
Gradient-Guided Decoupled Adaptation for Geospatial Vision-Language Models
Dongdong Wang, Deepak Balakrishnan, Ravi Srinivasan et al. (+1 more)
Existing geospatial
vision-
language models (Geo-VLMs) typically optimize diverse geospatial tasks through a unified multi-task adaptation paradigm without explicitly accounting for the heterogeneous optimization characteristics. Our empirical observations reveal heterogeneous gradient characteristics across tasks, including
vision-
language differences, intra-branch gradient relationships, and task interference, which hinder effective multi-task optimization. Motivated by these observations, we propose Gradient-Guided Decoupled Adaptation (G2DA), a gradient-aware optimization framework for multi-task Geo-VLM learning. G2DA first partitions tasks into
vision- and
language-centric groups through gradient-guided cross-modal decoupling. It then constructs modality-specific curricula based on task gradient similarity and employs bidirectional rehearsal to mitigate the recency effects introduced by sequential optimization. We evaluate G2DA on three Geo-VLM
benchmarks using six InternVL3 and Qwen3.5-VL variants, along with GeoChat and GeoLLaVA. Across all 24
benchmark-model combinations, G2DA consistently outperforms representative baselines, improving over the strongest competitor by 3.08, 4.30, and 2.81 percentage points on UrBench-MCQ, XLRS-Bench-Lite, and VRS-Bench-VQA, respectively. These results demonstrate the effectiveness of gradient-guided task organization for Geo-VLM adaptation.
Read the full abstract ▾
All authors Dongdong Wang, Deepak Balakrishnan, Ravi Srinivasan, Shenhao Wang
Artificial Intelligence · Cross-listed from cs.CV
REALIS: A Curated Dataset for Studying the Challenges of AI Image Detection
Aleksandr Gushchin, Khaled Abud, Georgii Bychkov et al. (+5 more)
AI-generated image detectors are often evaluated on
benchmarks where real and synthetic images differ in content, quality, or generation artifacts, allowing models to rely on
dataset-specific cues and fail on unfamiliar generators or processed images. Existing
datasets provide limited support for evaluating these challenges jointly across diverse visual content. We introduce REALIS, a
dataset of 1.43 million real and synthetic images generated by 42 modern text-to-image models, including the latest proprietary systems such as Nano Banana 2. REALIS combines
prompts derived from real images, quality filtering, and stratified sampling to reduce class-specific shortcuts while preserving content diversity. We further introduce REALIS-Expert, a stress-test subset for high-quality synthetic images, where real and generated samples are selected with closely matched semantic and visual characteristics. We also propose a robustness protocol covering 35 transformations at five severity levels to analyze detector behavior under image processing. Based on REALIS, our
benchmark evaluates pretrained detectors, fine-tuned models, and zero-shot
vision-
language models under generator and post-processing shifts. On the hardest processed split, the best pretrained conventional detector achieves 0.550 ROC-AUC, compared with 0.752 for the best REALIS-trained detector. REALIS provides a unified framework for measuring and improving the reliability of AI-image detectors under conditions that better reflect real-world use.
Read the full abstract ▾
All authors Aleksandr Gushchin, Khaled Abud, Georgii Bychkov, Ekaterina Shumitskaya, Artem Filippov, Sergey Lavrushkin, Dmitriy S. Vatolin, Anastasia Antsiferova
Artificial Intelligence
SkillVine: Agent Skill Evolution via Branching Exploration
Kaiwei Liu, Jiqian Dong, Liran Dong et al. (+7 more)
Agent skills encapsulate reusable procedural knowledge that enables LLM
agents to perform tasks, and they can be improved automatically using trajectories from interactions with the environment. This is the classic problem of skill
evolution. Existing approaches predominately follow a linear
evolution paradigm, in which updates are sequentially applied to the latest skill-library version. As a result, they inevitably fall into local optima, leaving many promising
evolution paths unexplored. We propose SkillVine, an automatic skill-
evolution framework that formulates skill
evolution as a graph search problem and employs a branching exploration strategy. Equipped with a trunk-branch collaborative searching mechanism, an intelligent parent-node selector, and an adaptive-granularity update rule, SkillVine achieves a balance between exploration and exploitation. We evaluate SkillVine on 5
benchmarks with two LLMs. Results show that SkillVine discovers better skill-library versions along branches than along the linear trunk and achieves the best test performance in nine of ten
benchmark-model combinations.
Read the full abstract ▾
All authors Kaiwei Liu, Jiqian Dong, Liran Dong, Shuai Mao, Mingming Zhao, Bufang Yang, Jie Chuai, Zhitang Chen, Guoliang Xing, Zhenyu Yan
Artificial Intelligence · Cross-listed from cs.CL
LLM Alignment–Utility Asymmetry under Semantic-Preserving Transformations
Mohan Li, Chengyu Yu, Francesco Sovrano et al. (+2 more)
Large
Language Model (LLM)
alignment is intended to ensure that models remain helpful and safe, but its stability under input distributional shift is not yet fully understood. Prior work shows that aligned models can fail under jailbreak
prompts, alternative encodings, and cross-lingual transfer, yet these failures are usually studied as attacks rather than controlled probes of
alignment generalization. Moreover, existing evidence is largely grounded in natural
language variation already represented during pretraining, leaving unresolved whether
alignment generalizes with semantic content or remains tied to superficial surface patterns. In this paper, we study this question using synthetic semantic-preserving transformations that are rule-based and invertible, preserving task-relevant meaning while shifting inputs beyond standard linguistic variation. Across four open-weight and four commercial models, under both fine-tuning and in-context learning, we use these transformations as a probe of
alignment generalization and identify an empirical pattern we term
Alignment–utility asymmetry: once models can operate effectively on transformed inputs, task utility is often substantially retained while
alignment failure increases more sharply. For example, adapted GPT-4.1 mini shows only limited utility degradation under transformation while its harmful rate rises from 13.3 to 74.3; Gemini 3 Flash similarly retains near-original utility while its harmful rate increases from 2.3 to 43.0. Taken together, these results suggest that semantic-preserving distribution shifts can expose a recurring gap in how utility and
alignment generalize in current LLMs.
Read the full abstract ▾
All authors Mohan Li, Chengyu Yu, Francesco Sovrano, Marc Langheinrich, Martin Gjoreski
Artificial Intelligence
MassAlloc Attention: Let Attention Allocate Its Own Compute
Jingze Shi, Zhangyang Peng, Xianduo Li et al. (+6 more)
FullAttn often assigns negligible normalized mass to much of the causal score space, yet dense kernels execute the complete post-score path after forming each QK tile. We introduce MALA, a fused
attention primitive that preserves score access to every legal causal interaction and uses normalized contribution to allocate post-score computation. Forward uses its evolving online-softmax normalizer, while backward reuses the finalized normalizer to derive nested retained support using only standard
attention state. A common tolerance governs training and inference, allowing for adaptive retention of the work. MALA reduces low-contribution post-score computation. A matched-work study at 8K isolates the benefit of distribution-adaptive allocation: under exactly matched total post-score work, MALA approaches a per-instance reference-mass oracle, with mean omitted mass of 0.0188% versus 0.0182%. Across context lengths from 1K to 32K
tokens, the same tolerance maintains low output and gradient errors relative to the reference. Across a broader controlled associative-recall comparison, MALA closely tracks FullAttn as context grows, reaching 89.67% accuracy at 8K compared with 89.97% for FullAttn. In an
attention-operator
benchmark at 128K
tokens with tensor parallelism, MALA reduces forward and backward latency during training by 2.2x and 3.0x and decoding latency during inference by 1.6x relative to FullAttn. Across scaling-law training from 0.6B to 14B parameters, MALA closely tracks FullAttn in perplexity while reducing total training FLOPs. The resulting 14B models and 32B models from separate continued training achieve comparable knowledge,
reasoning, and long-context
retrieval scores to FullAttn. These results indicate that allocating post-score computation according to normalized
attention contributions can retain the evaluated capabilities of FullAttn while reducing
attention computation.
Read the full abstract ▾
All authors Jingze Shi, Zhangyang Peng, Xianduo Li, Yanlin Qi, Xiaotian Lin, Haoxian Chen, Liangdong Wang, Guang Liu, Yuyu Luo
Artificial Intelligence · Cross-listed from cs.CR
Learning to Refer: Client-Resolved Generation for Privacy-Aware Language Models
Jeongho Yoon, Chanhee Park, Yongchan Chun et al. (+5 more)
Cloud-based large
language models (LLMs) require users to disclose plaintext data to service providers, creating
privacy risks in sensitive domains. Existing
privacy-preserving approaches often trade utility for protection, incur substantial computational or communication overhead, remain vulnerable to reconstruction from intermediate representations, or protect only a subset of the training and inference pipeline. We introduce Client-Resolved Generation (CRG), a genera- tion interface that separates server-side generation from the lexical realization of input-derived content. The client transmits only pooled and noise-perturbed rep- resentations, while input-derived output content is represented using request-local positional references and resolved to its original strings only on the client. This interface protects private input and input-derived output content during both train- ing and inference while allowing the service provider to keep its proprietary model parameters hidden from the client. At the same time, exact lexical reuse remains possible without directly exposing the reused content on the provider-visible gen- eration path. We evaluate CRG on medical and document-grounded QA, sensi- tive identifier transfer, and tool calling, together with reconstruction and raw-logit leakage analyses. On SealTools, CRG improves complete-call exact match from 57.3% to 79.9% over the input-
privacy framework PPFT, with larger gains as more required output content can be resolved through references. Together, these results show that CRG provides a practical interface for
privacy-sensitive cloud LLMs by reducing plaintext exposure across both input and output pathways while preserv- ing task utility and server-side model confidentiality.
Read the full abstract ▾
All authors Jeongho Yoon, Chanhee Park, Yongchan Chun, Duong Tuan Thanh, Sungbin Han, Chanjun Park, Hyeonseok Moon, Heuiseok Lim
Artificial Intelligence
CoWindow Attention: Full Causal Coverage Is a Collective Property
Jingze Shi, Zhangyang Peng, Xianduo Li et al. (+6 more)
FullAttn repeatedly exposes the complete causal history to every
attention head, creating substantial redundant computation and
memory traffic even with IO-efficient dense kernels. We introduce CoWA, a structured
attention architecture that distributes access to the causal history across KV heads. All heads share near-diagonal and prefix-sink windows, while complementary long-range windows partition the remaining history. Their union provides full causal coverage although each head attends sparsely to distant
tokens. This position-defined
attention pattern requires no learned router or indexer, is used consistently during training and inference, and aligns with KV-head tensor parallelism. A window-matched ablation at 8K isolates the effect of complementary long-range allocation: CoWA with 100% collective coverage reaches 89.73% accuracy, compared with 89.97% for FullAttn, while duplicated long-range windows perform substantially worse. Across a broader controlled associative-recall comparison with matched
token budgets, CoWA closely tracks FullAttn as the context grows, whereas other sparse patterns lose a substantial fraction of the associations. In an
attention-operator
benchmark at 128K
tokens with tensor parallelism, CoWA reduces forward and backward latency during training by 7.4x and 8.6x and decoding latency during inference by 3.0x over FullAttn. Its per-rank peak operator
memory matches FullAttn during training and is 7.6x lower during decoding. Across scaling-law training from 0.6B to 14B parameters, CoWA closely tracks FullAttn in perplexity while reducing total training FLOPs. The resulting 14B models and 32B models from separate continued training achieve comparable knowledge,
reasoning, and long-context
retrieval scores to FullAttn. These results show that full causal coverage can be a collective property of the head ensemble rather than a duplicated property of every head.
Read the full abstract ▾
All authors Jingze Shi, Zhangyang Peng, Xianduo Li, Yanlin Qi, Xiaotian Lin, Haoxian Chen, Liangdong Wang, Guang Liu, Yuyu Luo
Artificial Intelligence
Despite Instructions: Frontier Agents Improvise Covert Channels at Test Time
Jacob Dineen, Silei Ren, Muhao Chen et al. (+2 more)
In security-sensitive applications,
language-model
agents are often required to coordinate without disclosing confidential information. Yet repeated interactions may also let ordinary messages acquire shared private meaning. We study a repeated game with pairs of models in which the sender model observes one of four secret states and selects one of four summaries of the same public report, while the receiver model tries to infer the secret state. We find that model pairs can learn to communicate the secret using only one bit of feedback indicating whether the receiver inferred it correctly. This learning occurs during inference with fixed parameters and no supplied codebook or encoding examples. The effect also persists when
agents generate their own free-form updates in a simulated incident-response task. Across ten independent games, pairs of GPT-5.6 Sol
agents reach 98.8% final accuracy, compared with 25% chance, despite explicit instructions prohibiting disclosure and a monitor that screens each message without access to the
agents' interaction histories. The same interactions that help
agents cooperate can therefore allow confidential information to pass through messages intended for legitimate coordination.
Read the full abstract ▾
All authors Jacob Dineen, Silei Ren, Muhao Chen, Dan Roth, Ben Zhou
Artificial Intelligence
CAIRN: Dynamic Fact-Intent DAGs for Multi-Agent Exploration
Zuyao Xu, Yuyang Jia, Junwei Guan et al. (+3 more)
LLM-powered autonomous systems have demonstrated promising capabilities in mathematical
reasoning, engineering, and cybersecurity. Yet how to organize these systems for effective, reliable, and sustained performance remains an open question. In this paper, we present CAIRN, a fact-intent-driven multi-
agent paradigm for goal-directed exploration. CAIRN represents observations and planned investigations as a dynamic directed acyclic graph (DAG). A reasoner interprets facts to propose intents, which workers execute to produce new facts. Each intent references its supporting facts and defines a potential exploration branch. The persistent graph preserves goals, dependencies and findings across workers, supporting knowledge reuse and parallel exploration. The graph also makes execution trajectories traceable and auditable, providing a basis for human verification and intervention. We evaluate CAIRN across cybersecurity and mathematical
reasoning tasks, examining task success, time to solution, and
token consumption. DAG-based coordination can incur higher
token costs with no observable performance gains on tasks that require little effort. However, on high-effort tasks (at least 1M
tokens), we observe faster solutions in 76.5% of cases, with speedups of up to 3.08x. Moreover, as task effort increases, these time gains become more pronounced while relative
token overhead declines, highlighting the potential of DAG-guided parallel exploration.
Read the full abstract ▾
All authors Zuyao Xu, Yuyang Jia, Junwei Guan, Xiang Li, Kaiwen Shen, Zhiqiang Dong
Artificial Intelligence
Flat-Consensus Diffusion for Robust Data Reshaping under Noisy Evaluator
Hongyu Cao, Kunpeng Liu, Fei Xie et al. (+1 more)
Data shape determines how features are structured, how patterns are separated, and how distributions cover the underlying domain. Poor data shape can make models learn noise rather than generalizable structure. This paper studies robust feature-centric data reshaping: generating feature transformations that remain useful, stable, and reproducible under noisy evaluation and imperfect data conditions. We view reshaping operation sequence search as reward-guided
diffusion generation, and robust reshaping as searching for regions in the latent reward landscape rather than isolated high-reward transformations. The key challenge is dual instability: noisy evaluators distort local reward guidance, while
stochastic generative trajectories can converge to inconsistent solutions. We propose FCDiff, a flat-consensus
diffusion framework that addresses both failures through a micro-macro decomposition. The micro layer replaces point-estimate reward guidance with Gaussian-smoothed, Monte Carlo averaged gradients, steering generation toward locally flat reward regions. The macro layer aggregates independently guided trajectories with a weighted Frechet-mean barycenter, selecting consensus-supported basins and filtering
stochastic outliers. Across an 8-
dataset headline cohort under heavy-tailed evaluator noise, FCDiff attains the best aggregate rank on lower-tail reliability and robustness against both search-based AutoFE and robustness-oriented generative baselines, with statistically significant accuracy gains over every generative baseline. Our results show that robust data reshaping requires searching for flat, consensus-supported regions rather than sharp single-trajectory optima.
Read the full abstract ▾
All authors Hongyu Cao, Kunpeng Liu, Fei Xie, Sandip Ray
Artificial Intelligence
IGSD: Environment-Verified Hindsight Self-Distillation for Search Agents
Angqing Jiang, Gaoming Zhang, Chaoqun Zhang et al. (+5 more)
On-policy self-
distillation densifies
agent training without external teachers: a policy conditioned on privileged hindsight provides step-level guidance for its own unprivileged rollouts. For search
agents, however, hindsight can make the teacher prefer a query that does not improve
retrieval from the student's state. Existing methods either distill this preference directly or filter it with model-internal scores, but neither strategy verifies the query's executed
retrieval consequence. We propose Information-Gain-Gated Self-
Distillation (IGSD), which verifies on-policy
token proposals with environment feedback before distilling them. Treating each query
token as a micro-action, IGSD completes the teacher's
token proposal and the student's sampled
token into matched queries and executes both from the same failed state with the same retriever. Shared counterfactual controls account for query-conditioned shifts in answer likelihood, so their difference, the executed paired information gain, provides a relative utility contrast for the retrieved documents. IGSD uses this contrast as a positive-only soft weight for candidate-pair
distillation, while leaving the GRPO objective unchanged and confining verification to training. Across seven single-hop and multi-hop QA
benchmarks, IGSD reaches macro-average exact-match accuracies of 42.8% and 47.0% with 3B and 7B policies, respectively, without inference-time verification. These results support environment-verified hindsight as an effective approach to reliable action-level supervision for search
agents.
Read the full abstract ▾
All authors Angqing Jiang, Gaoming Zhang, Chaoqun Zhang, Jianchun Song, Liyuan Kong, Kena Qi, Wei Lin, Defu Lian
Artificial Intelligence
World Agent: Can Language Models Keep a World Running?
Weixing Chen, Weipeng Zhang, Nan An et al. (+2 more)
World models are moving from generating realistic frames to generating playable worlds, yet whether a delivered world can keep running is not tested anywhere. Existing evaluations stop at generation, at delivery, or at single-step transitions, and each stops at a different point along the way. Correct local state transitions or intermediate outcomes do not guarantee a correctly organized causal event flow. We propose the world
agent task, which moves the evaluation point of world generation from the moment of delivery to the continued operation that follows. In this task, a model is not asked to generate a world. It is held responsible for keeping the world running, which requires coordinating events and carrying forward their consequences to constrain subsequent
evolution. We instantiate the task in WorldAgent-
Benchmark with two complementary tracks. In the maintenance track, the model must ground the events of a continuous narrative into correct transitions of the explicit world state while respecting causal, temporal, and concurrency constraints. In the deduction track, the model must predict how the world will evolve under partial observations and act toward a goal. The maintenance track combines LLM-assisted semantic judgments with programmatic validation and scoring, while the deduction track is evaluated entirely programmatically. Individual judgments are auditable against world states and execution logs, and scores can be recomputed from the saved judgments and execution records. Across 8 models, scores decline steadily as pre-built structure is removed from the world, and causal-relation checking is the weakest component for every model. The
benchmark makes the continued operation of a world measurable and distinguishes local completion from failures in event organization. Code and
dataset will be released on https://github.com/HCPLab-SYSU/WorldAgent-
Benchmark.
Read the full abstract ▾
All authors Weixing Chen, Weipeng Zhang, Nan An, Yang Liu, Liang Lin
Artificial Intelligence · Cross-listed from cs.CV
MM-OPD: Towards One More Bottleneck Between Perception and Reasoning
Jintao Tong, Yujing Lou, Zhanming Shen et al. (+6 more)
Recent
multimodal large
language models (MLLMs) advance visual
reasoning by strengthening both perception and
reasoning, implicitly assuming a process that transitions seamlessly from perception to
reasoning. However, we observe a counterintuitive phenomenon that challenges this assumption: holding the model, question, and decoding fixed, we replace images with their caption or code representations (symbolic views), which seems to be redundant given the clear image structures, but the performance surprisingly improves by 10.2% to 23.6% across model scales and
datasets. We term this performance gap as the Symbolic Visual Gap and then take a closer look at it. Through experiments, we find that although the visual evidence can already appear in the
reasoning trace for the image-input model, the symbolic-view-input model shows much higher
attention to the correct evidence than the image-input model. This suggests that despite good capabilities from current works in perception and
reasoning themselves, another bottleneck exists between perception and
reasoning in selecting perceived visual information as appropriate evidence for subsequent
reasoning. To handle this bottleneck, since the symbolic view steers
attention toward correct evidence and is readily obtained at scale, it provides supervision for evidence selection without manually labeled evidence. Building on this, we introduce MM-OPD, a
multimodal on-policy self-
distillation framework for symbolic-to-visual correction that transfers guidance from symbolic-conditioned behavior to the image-conditioned policy through residual
token-level targets, steering the model toward correct visual evidence. Experiments across
benchmarks and model scales show that MM-OPD improves a broad range of
multimodal abilities, with gains in visual perception, chart and document understanding, mathematical
reasoning, and general VQA.
Read the full abstract ▾
All authors Jintao Tong, Yujing Lou, Zhanming Shen, Jiaqi Gu, Lubin Fan, Ruixuan Li, Yue Wu, Jieping Ye, Yixiong Zou
Artificial Intelligence
Dude, Where's My State? Execution Information Requirements for Stateful Agents
Nikita Mehrotra, Ashish Tiwari, Priyanshu Gupta et al. (+1 more)
Long-running
agents must preserve information that later steps depend on. We introduce the Execution Information Requirement (EIR), a lower bound on the information that must remain accessible for correct completion under specified task and access conditions. We develop LACUNA, a framework that generates tasks with known dependencies and varies information demand, retention, and recovery separately from the difficulty of individual operations. Across four models, restoring a missing result raises accuracy on affected recall steps to 100%, compared with 0% for equal-length irrelevant information. Sufficient storage alone does not ensure success: retention policies can discard required results, errors can propagate through later computations, and
agents can stop before recovery is complete. We also introduce VESTIGE, which uses
agent execution traces to construct semantic graphs and measure information demand for real tasks. Across 72,562 software-
agent trajectories, VESTIGE reveals a steeper distance-related decline in solution-relevant rereading for failed runs (RR 0.951 per distance doubling), while adjusted peak demand alone is not associated with failure. Together, these contributions support evaluating whether
agents preserve and recover the information their tasks require.
Read the full abstract ▾
All authors Nikita Mehrotra, Ashish Tiwari, Priyanshu Gupta, Sumit Gulwani
Artificial Intelligence
PINNMorph: Evolving Online Adaptation Policies for Physics-Informed Neural Networks
Xu Yang, Mingyang Yu, Jun Zhang et al. (+2 more)
Physics-informed neural networks (PINNs) provide a learning-based framework for solving partial differential equations (PDEs), yet their training behavior can change substantially throughout optimization. Residual distributions, gradient interactions, regional learning difficulty, and model-capacity requirements may evolve over time, while the network architecture and major training mechanisms are typically determined before training. We propose PINNMorph, an online PINN adaptation framework based on large
language model (LLM)-guided policy
evolution. PINNMorph maintains a population of state-conditioned adaptation policies that map execution diagnostics to controlled interventions over
topology modification, additive representation augmentation, objective balancing, gradient handling, adaptive sampling, and optimizer-phase control. At each intervention opportunity, candidate programs are instantiated from the current policy population, selected according to the observed training state, and applied directly to the PINN under training. The resulting model inherits its existing parameters and training state and continues optimization along the same trajectory. Execution outcomes are subsequently used to evaluate interventions and evolve the policy population. Unlike pre-training architecture search or fixed adaptation rules, PINNMorph jointly adapts the current PINN and the policies governing its interventions using feedback from actual training. Experiments on 13 PDE
benchmarks show that PINNMorph achieves lower solution errors than SA-PINN, ConFIG, RoPINN, HARMONIC, and PINNsAgent across all evaluated problems. Ablation studies further examine the effects of online adaptation, state-conditioned intervention selection, and execution-feedback-driven policy
evolution.
Read the full abstract ▾
All authors Xu Yang, Mingyang Yu, Jun Zhang, Keqian Li, Jing Xu
Artificial Intelligence · Cross-listed from cs.LG
The GUI Is Not the State: Diagnosing State Aliasing in GUI World Models
Dongsheng Liu, Chao Jin, Wenkui Yang et al. (+7 more)
GUI World Models (GUI-WMs) are increasingly used to predict future states for
agent planning and simulation, yet most existing formulations condition only on the current GUI observation and action. We identify state aliasing, where the vis- ible interface omits transition-relevant environment state, so identical observable conditions can correspond to different valid futures. To diagnose this failure mode, we introduce StateAliasBench, a diagnostic
benchmark that explicitly isolates such ambiguities via strict pairing. We further propose lightweight predictive- state recovery that infers structured state from history and augments otherwise frozen GUI-WMs through a deterministic state interface. Family-specific special- ists provide state recovery across heterogeneous state types, and multi-teacher dis- tillation consolidates them into a single unified
estimator. Experiments show that existing GUI-WMs exhibit systematic failures under observation-only condition- ing, while predictive-state augmentation substantially restores state-sensitive pre- diction across evaluated WMs, preserves generative fidelity, and improves down- stream performance of GUI
agents on AndroidWorld. These results suggest that reliable GUI world modeling should account not only for what is visible, but also for the hidden transition state that determines what happens next.
Read the full abstract ▾
All authors Dongsheng Liu, Chao Jin, Wenkui Yang, Hejin Wang, Junwei Yang, Zeren Zhang, Ziwei Chen, Huaibo Huang, Jie Cao, Ran He