Artificial Intelligence
MetaBench-Harness: Unlocking End-to-End Optimization of Benchmark Harnesses
Xuanjun Chen, Hua-Hsuan Chen, Wei-Chung Lu et al. (+3 more)
Rapid progress in Large
Language Models (LLMs) is saturating static
benchmarks faster than they can be designed. While existing automated
evolution frameworks attempt to generate harder questions by perturbing individual tasks, they remain constrained by rigid, hard-coded generation rules. Moving beyond the
evolution of isolated tasks, we propose to optimize the
benchmark generation workflow itself end to end with MetaBench-Harness, a dual-loop search framework. Specifically, the inner loop utilizes a
benchmark harness to generate a new
benchmark in each round, while the outer meta-harness orchestration layer iteratively refines and searches over harness implementations based on historical
evolution trajectories. By applying MetaBench-Harness to the competitive programming CodeContests and Olympiad mathematics AIME-2024
datasets, we demonstrate that the evolved
benchmarks are challenging and discriminative for frontier models. Trajectory and quality analyses verify that MetaBench-Harness enables multi-dimensional
evolution, steadily improving
evolution reasonableness,
benchmark competency, and evaluator robustness across successive rounds. Furthermore, case studies reveal its effective utilization of diverse difficulty levers to reframe problems and elevate required capabilities. Ultimately, this work provides a solution to the pressing challenge of
benchmark saturation.
Read the full abstract ▾
All authors Xuanjun Chen, Hua-Hsuan Chen, Wei-Chung Lu, Yinghao Ma, Jyh-Shing Roger Jang, Hung-yi Lee
Artificial Intelligence · Cross-listed from cs.LG
Decoupling Token Roles in Autoregressive Pretraining
Suqin Yuan, Runqi Lin, Kevin Qinghong Lin et al. (+4 more)
Autoregressive pretraining increasingly draws on heterogeneous data, making it important to understand how a model learns from an individual
token. The next-
token prediction objective naturally identifies a
token's contribution with its own loss. However, each
token is not only a prediction target but also context for what follows. Using controlled corruption, we decouple these two roles and find a reversal: making a noisy
token easier to predict reduces its damage as a target but increases it as context. The same decoupling helps explain text generated by
language models: generation selects each
token by its fit to the prefix, while its role as context is never tested against an independently determined continuation, because that continuation is generated to fit it. At known corrupted positions, acting through the context can reduce damage that removing the
token's own loss does not. Understanding and controlling what a model learns from a
token therefore requires decoupling its roles.
Read the full abstract ▾
All authors Suqin Yuan, Runqi Lin, Kevin Qinghong Lin, Junchi Yu, Lei Feng, Chris Russell, Tongliang Liu
Artificial Intelligence · Cross-listed from cs.HC
DataMagic: Authoring Data Videos through Declarative Multi-Agent Orchestration
Yupeng Xie, Zhenyang Wang, Liangwei Wang et al. (+3 more)
Data
videos communicate data insights through dynamic charts, voice narration, and synchronized animations, and have become a widely adopted form of data storytelling. However, producing them requires expertise in data analysis, narrative design, and
video editing. Static visualization tools lack narrative and animation capabilities; authoring tools rely on pre-prepared charts rather than raw data; and pixel-level models generate
videos end-to-end but cannot guarantee data accuracy or provenance. End-to-end automatic generation faces two core challenges: how to uniformly represent charts, narration, and animations together with their temporal relationships, and how to efficiently search a vast design space for narrative-coherent compositions. We present DataMagic, which authors data
videos from raw tabular data through declarative multi-
agent orchestration. First, the declarative specification DVSpec unifies charts, narration, and animations with data-bound references and declarative synchronization, ensuring data provenance and automatic audio-visual
alignment. Second, a "Generate-then-Orchestrate" multi-
agent strategy generates candidate scenes in parallel and then optimizes narrative coherence through global orchestration. DVSpec provides a shared state for three complementary interaction modes, bridging full automation with fine-grained human control. Evaluations on 109 real-world samples show that even the most advanced LLM (e.g., GPT-5) achieves only 2.13/5 with execution success rates between 48.62% and 86.24%; DataMagic improves quality to 3.89 (+83%) with success rates above 95%, with the most significant gains in animation and narrative dimensions. A user study shows that, compared to a conversational LLM workflow, DataMagic improves creation efficiency (79.7% reduction in task time) and reduces perceived cognitive load. Project page: https://github.com/HKUSTDial/DataMagic.
Read the full abstract ▾
All authors Yupeng Xie, Zhenyang Wang, Liangwei Wang, Jiayi Zhu, Zhouan Shen, Yuyu Luo
Artificial Intelligence · Cross-listed from cs.CR
Evaluating System One Models for Agent Security Decisions: Reliability, Calibration, and Selective Automation
Yixuan Liu
Agentic software connects
language models to tools that modify files and interact with external services. Developers use model-based security checks to screen external content and user requests before
agents act. System One models answer developer-defined questions with probabilities over predefined answers such as safe or unsafe, but classification accuracy alone does not establish whether these probabilities support reliable automation. We evaluate Jev, Laya, Decider, and Bespoke Nimble against specialized classifiers and
language-model judges across
prompt-injection detection, interaction-risk judgment, and harmful-request screening, examining decision accuracy, calibration, and selective automation. (1) High overall accuracy and low average calibration error can conceal attacks classified as safe with high confidence within particular groups. Developers should test candidate models on the intended security task and examine errors within relevant attack groups. (2) Policies selected under strict miss limits allow few test inputs automatically, and choosing allow and block thresholds separately increases automation mainly through additional blocks. Developers should verify limits on missed unsafe inputs and blocked benign inputs on independent data, and report the allowed and blocked fractions separately. (3) Comparing predictions on the same inputs shows that a
language-model judge can detect unsafe inputs missed by a System One model, but can also repeat the System One model's confident mistakes and falsely flag benign inputs. Developers should test which missed unsafe inputs their review rule forwards, then measure the review model's misses and benign false alarms on those inputs.
Read the full abstract ▾
Artificial Intelligence
COEVO: Co-Evolving Context and Parameters for Recursive Self-Improvement
Siwei Chen, Xinping Bao, Xinyu Cai et al. (+3 more)
Recursive self-improvement (RSI) seeks to move large
language models beyond static training pipelines toward systems that can participate in improving their own future behavior. Existing approaches largely follow two directions: updating model parameters through online learning, or improving the external context through search, reflection, and
prompt optimization. Although both mechanisms can support continued improvement, they are typically studied independently. This separation overlooks an important interaction: the context shapes the experience from which a model learns, while an evolving model may interpret and utilize the same context differently over time. We therefore formulate RSI as a problem of parameter–context co-
evolution, where model parameters and the learning context adapt within a shared feedback loop. We introduce COEVO, a framework that updates model parameters from on-policy experience while adapting contextual guidance according to the state of the evolving policy. Policy
entropy and
prompt-conditioned
attention are used as complementary signals to guide this adaptation. Experiments show that COEVO consistently improves task performance over fixed-context reinforcement learning and produces policies that are more robust to changes in system
prompts. More broadly, our results suggest that external context should be viewed not merely as a fixed interface to a large
language model, but as an adaptive component of recursive self-improvement.
Read the full abstract ▾
All authors Siwei Chen, Xinping Bao, Xinyu Cai, Yuan Cao, Wan Jiang, Shaohong Chen
Artificial Intelligence
CoViST: Visual Token Compression via Composable States
Qi Zhang, Xiandong Meng, Ronggang Wang et al. (+1 more)
Visual
token compression lowers the inference cost of
vision–
language models by representing images with fewer
tokens. However, most existing methods compress visual
tokens to a reduced set, leaving the amount of visual evidence represented by each
token and its original spatial context implicit. Therefore, the compressed representation does not explicitly encode how much visual information each representative carries or where it lies in the original image. This limitation arises even after a single reduction and becomes more pronounced when compression is repeated across decoder layers. To address this issue, we propose CoViST, a training-free framework that represents a compressed image as a composable visual state. Specifically, the state combines representative features with original positions, effective contribution weights, and reusable selection metadata. CoViST constructs this state through coverage-guided selection and conservation-based contribution composition, and explicitly incorporates its contribution and positional information into decoder
attention. Each component of the state retains its interpretation under successive reductions, enabling the same formulation to support both fixed compression before prefill and progressive compression within the decoder. Experimental results on seven LLaVA-1.5-7B
benchmarks show that CoViST-Fixed retains 99.9%, 99.5%, and 98.1% of uncompressed performance at 192, 128, and 64
tokens, respectively, and CoViST-Pro retains 99.8%, 99.9%, and 99.1% at the corresponding layer-average budgets, outperforming state-of-the-art methods under their respective budget settings. Code will be released publicly.
Read the full abstract ▾
All authors Qi Zhang, Xiandong Meng, Ronggang Wang, Siwei Ma
Artificial Intelligence
Cross-modal Translation via Conditional Latent Denoising for Video Deepfake Detection
Xinzhe Li, Youzhi Tu and Kong Aik Lee
The growing threat of
video deepfakes necessitates
multimodal detection. Beyond serving as independent indicators of authenticity, audio and visual signals have intrinsic dependencies that also provide an essential criterion for detection. Previous methods often overlook the cross-modal correspondences, hindering information transfer between domains and leaving crucial detection cues unexplored. To address this challenge, we propose a framework called Cross-modal Translation via Conditional Latent Denoising (CTCLD) for
video deepfake detection. It connects the distinct distributions of heterogeneous modalities in latent spaces, enabling smooth cross-domain information transfer to improve detection performance. We first establish a
Bayesian foundation by decomposing the audio-visual joint distribution. Subsequently, CTCLD translates both modalities via bidirectional latent denoising conditioned on each other, effectively capturing subtle inconsistencies in the manipulated signals. Experimental results demonstrate that the proposed CTCLD enables comprehensive domain
alignment, resulting in a robust
video deepfake detection approach with competitive performance.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
AutoHGNN: Robust and Efficient Neural Architecture Search for Hypergraph Neural Networks
Sirui Li, Pietro Liò b, Xinsheng Li et al. (+2 more)
Hypergraph neural networks have achieved significant success in recent years. However, manual architecture crafting is labor-intensive and often fails to capture complex, higher-order relations, making the automation of hypergraph neural network structure design crucial. To improve the automation and adaptability of hypergraph learning, this paper proposes AutoHGNN, a neural architecture search framework tailored for hypergraph neural networks. First, we introduce a Hyper-Interaction Module (HIM) into the search space to address the mismatch between conventional graph neural network designs and hypergraph data. Second, we propose Hypergraph Stable Topological Distance (HyperSTD) as a structural selection criterion to identify architectures that best preserve the intrinsic structural affinities of the original hypergraph during differentiable search. Extensive experiments on various
benchmark datasets demonstrate that AutoHGNN consistently outperforms manually designed and automatically searched baselines in classification accuracy and time efficiency, proving that the discovered architectures are significantly more effective.
Read the full abstract ▾
All authors Sirui Li, Pietro Liò b, Xinsheng Li, Baisong Liu, Chengbin Peng
Artificial Intelligence · Cross-listed from cs.LG
Beyond Timestamps: Decision-Aligned On-Policy Distillation for Long-Horizon Agents
Mingju Chen, Can Lv, Jinrong Liu et al. (+3 more)
Reinforcement learning with verifiable rewards (RLVR) often relies on sparse outcome rewards, providing coarse supervision for long-horizon
agents. On-policy self-
distillation (OPSD) complements this signal with dense privileged feedback. However, we identify Decision–Timestamp Mismatch: privileged guidance may be misaligned with the student's functional decision because the corresponding decision can occur at a different timestep, while the student's decision itself may span multiple timesteps rather than being tied to a single timestamp. Thus, timestamp-local supervision can misalign both the context and the temporal scope of credit. To address this mismatch, we introduce AlignOPSD, following the principle of aligning supervision before assigning credit. Decision-Aligned Supervision Rectification re-scores the same student-sampled response in functionally matched contexts across sibling rollouts to calibrate local teacher evidence. Semi-Markov Hierarchical Credit Assignment then derives variable-duration decision spans from correspondence changes and uses rectified evidence to allocate outcome-grounded credit across spans and their constituent turns. We evaluate AlignOPSD with Qwen2.5-3B and Qwen2.5-7B on ALFWorld, WebShop, and Search-QA against representative baselines. AlignOPSD outperforms both GRPO and StepOPSD across all eight backbone–aggregate-metric comparisons, improving on GRPO by 5.5–8.7 % and ranking first in six. Additional analyzes examine the two
alignment stages and hyperparameter sensitivity between tasks. Our code is avaliable at https://github.com/mingju-c/Align-OPSD
Read the full abstract ▾
All authors Mingju Chen, Can Lv, Jinrong Liu, Huan Zhang, Heng Chang, Shiji Zhou
Artificial Intelligence · Cross-listed from cs.CL
From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
Siwei Chen, Yuxiang Wan, Yifan Yu et al. (+1 more)
Diffusion language models (DLMs) can accelerate generation by predicting multiple
tokens in parallel, but there is a mismatch between how these
tokens are predicted and how they ultimately contribute to generation. Parallel predictions can hardly condition on the
tokens selected earlier within the same block, even though their validity depends on this realized prefix. Under the popular proposal-verification decoding, this mismatch makes errors highly asymmetric: an early rejection prevents all subsequent proposals from contributing decoding progress. We introduce BRISK-DLM, a framework that addresses both mismatches by optimizing proposal learning and selection for verified progress. BRISK-DLM trains on self-generated sequences, using risk-reward weighting to dynamically prioritize positions by their impact on verified progress and decoding cost. During inference, a lightweight prefix-conditioned corrector reranks existing candidates using previously selected
tokens and preferences distilled from the model's own verifier. The corrector reuses the backbone's parallel representations and requires no additional backbone evaluation, while fused execution keeps its overhead small. BRISK-DLM improves end-to-end throughput by up to 37.4% while preserving task quality, establishing a new quality-throughput frontier for DLM generation.
Read the full abstract ▾
All authors Siwei Chen, Yuxiang Wan, Yifan Yu, Fan Lai
Artificial Intelligence · Cross-listed from cs.CL
NLPG: Natural-Language Policy Gradients for Self-Evolving Language Agents
Xu Liu, WenZhang Wei, Jun Cao et al. (+4 more)
Large
language model
agents increasingly rely on compound programs for
retrieval, tool use,
reasoning, and verification, yet their failures often arise from local procedural decisions. Existing reinforcement-learning and
prompt-optimization approaches typically rely on scalar rewards or repeatedly modify entire
prompts, making it difficult to capture and reuse procedural improvements while preserving a frozen
agent. To address this problem, We propose Natural-
Language Policy Gradients (NLPG), an external policy-
memory method for improving a fixed
agent without changing its model parameters or program structure. NLPG diagnoses execution traces, propagates downstream feedback backward through the module graph, and converts recurring failures into route-local natural-
language corrections that are aggregated into bounded policy updates for subsequent executions. Across six
benchmarks covering
memory,
reasoning, instruction following, and evidence verification, NLPG also outperforms the strongest listed baseline for each
benchmark by 8.71 percentage points on average. These results provide evidence that evaluated procedural experience can be transformed into local and interpretable policy updates, enabling continual improvement of frozen
agents.
Read the full abstract ▾
All authors Xu Liu, WenZhang Wei, Jun Cao, Dehua Peng, Huan Chen, Zhipeng Gui, Huayi Wu
Artificial Intelligence · Cross-listed from cs.RO
Seungyeon Kim, Junhoo Lee, Minkyu Kim et al. (+2 more)
A central goal in robotics is to enable manipulation across changing tasks and environments.
Vision-
language-action (VLA) models provide broad manipulation capabilities but can struggle when execution requires diagnosing failures and adapting behavior. Strong
agents can discover effective interventions through interaction with these policies. We propose Recursive Harness
Distillation to accumulate this experience as reusable guidance across
agents. A strong
agent distills its experience into a playbook for a light
agent, then recursively refines the playbook using the light
agent's execution feedback. The resulting playbook enables
agents to reuse accumulated intervention knowledge in new task instances without updating model parameters. In real-world manipulation, the harness improves success from 37.3% to 64.0%. On SimplerEnv Bridge, the light
agent with the playbook achieves 66.7% success, compared with 41.7% for the GR00T-only baseline, and outperforms the strong
agent without a playbook. The same playbook also benefits the strong
agent, which reaches 79.2% success. These results demonstrate the feasibility of harness
distillation for robotics: intervention experience can be accumulated, refined through execution, and reused across
agents to improve manipulation.
Read the full abstract ▾
All authors Seungyeon Kim, Junhoo Lee, Minkyu Kim, Baekseung Kim, Nojun Kwak
Artificial Intelligence · Cross-listed from cs.SD
What Survives the Codec Shift: Pooled No-Vocals Residuals for Speech Deepfake Detection
Jiajun Xu, Menglu Li and Xiao-Ping Zhang
The transition from vocoder-based to neural-codec
speech synthesis makes generalization more difficult for
speech deepfake detectors, particularly those relying on
speech-oriented representations. It remains unclear which acoustic representations retain discriminative information when the generation mechanism changes. We therefore compare 12 acoustic representations using a shared low-capacity linear classifier to identify effective evidence under codec shift. The analysis shows that hierarchical XLS-R leads on the pooled test set, while pooled no-vocals residual statistics perform best on the unseen-codec condition, revealing complementary behavior across generation conditions. Building on this finding, we propose MN-P, a dual-view detector that integrates an utterance-level pooled no-vocals representation with
token-level XLS-R features through adaptive gating. The proposed MN-P reduces EER by 54.2% overall and by 60.9% on the codec-unseen condition relative to the best-performing retrained state-of-the-art system, with consistent gains across different detector backends. These results indicate that pooled no-vocals residual statistics provide effective complementary evidence for cross-generation
speech deepfake detection.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
API Secrets Should Never Become Tokens in the LLM's Vocabulary: A Threat Analysis of API Credential Handling in LLM Agent Systems and an Empirical Evaluation of a Vault-Mediated Execution Boundary
Patrick Kenney, Hadi Ahmadi, Denis Lusson et al. (+2 more)
Tool-using large
language model (LLM)
agents turn credential hygiene from a storage problem into an execution-security problem. A key pasted into a
prompt, or embedded in a system
prompt or tool configuration, crosses from an authentication boundary into a data pipeline, where it may persist in conversation history, logs,
memory stores, generated code, and error payloads.
Prompt injection and excessive agency then convert passive disclosure into unauthorized action. This paper formalizes the credential-exposure threat chain for agentic systems; synthesizes evidence from a platform secret-store incident, vendor-reported secret-sprawl measurement, and OWASP and NIST guidance; and describes a vault-mediated execution architecture in which the model selects a connector identifier while a trusted request boundary supplies authentication. We evaluate a production implementation, Corvic Security Vault, in two controlled black-box experiments. Across 16 probes spanning seven control domains, every probe met its expected outcome: an authenticated GitHub API request succeeded while the credential stayed absent from process environment values, caller-visible request headers, tested filesystem locations, three third-party echo services, and two unrelated API origins; both cloud instance-metadata endpoints were unreachable. We also report a negative result, a connector whose stored header mapping did not satisfy its provider's authentication contract, showing that centralized custody does not by itself guarantee correct configuration. Vault mediation removes several disclosure paths but is necessary rather than sufficient: least privilege, deterministic action authorization, human approval, telemetry redaction, and rotation remain independently required. The study is purposive and small, a functional security evaluation rather than a certification.
Read the full abstract ▾
All authors Patrick Kenney, Hadi Ahmadi, Denis Lusson, Donald Nguyen, Gurbinder Gill
Artificial Intelligence
DrafTS: Time-Aware Decomposition with Residual Correction for Time Series Modeling
Yiqiu Liu, Siru Zhong, Zhiguang Wang et al. (+2 more)
Real-world time series contain evolving underlying dynamics with irregular variations that lack stable temporal patterns and are often referred to as noise. Existing methods address this mixture by filtering frequencies or suppressing noisy observations. They either miss temporal
evolution or risk suppressing useful dynamics. We propose DrafTS, a model-agnostic framework that aims to reduce noise while preserving evolving dynamics through time-aware Decomposition with ResiduAl correction For Time Series. DrafTS uses features derived from instantaneous amplitude and frequency to guide decomposition into a primary component intended to capture underlying dynamics. A task-specific backbone models the primary component, while a lightweight correction module uses residual information to correct the backbone output. Across four time series modeling tasks, DrafTS improves six diverse backbones, demonstrating its effectiveness. Code is at https://github.com/Autumn61q/DrafTS
Read the full abstract ▾
All authors Yiqiu Liu, Siru Zhong, Zhiguang Wang, Qingsong Wen, Yuxuan Liang
Artificial Intelligence
DISCERN: Can AI Agents Work Like Scientists and Guide Discovery?
Nan Huang, Mario Tapia-Pacheco, Kun Zhou et al. (+4 more)
Reliable automated research requires
agents to vet data, verify analyses, and generate hypotheses grounded in trustworthy evidence, potentially reducing routine scientific workload while allowing scientists to focus on interpretation and discovery. Existing
benchmarks often only assess analytical task completion or hypothesis generation separately rather than testing whether reliable evidence supports valid and novel claims. We introduce DISCERN (Data Integrity and Scientific Capability: Evidence,
Reasoning, and Novelty), a controlled
benchmark on real, publicly available
datasets that evaluates three key levels of an automated research workflow. The first two levels test data integrity and analysis verification under confounds and tool traps, while the third tests hypothesis generation and revision under
adversarial review, including counterfactual cases in which evidence consistent with real data and documented scientific phenomena conflicts with established expectations, motivating alternative explanations and testable hypotheses. Across 203 tasks, eight life-science tracks, and eight models, DISCERN shows that strong aggregate performance can mask level-specific weaknesses.
Agents earn perfect scores in only 60.8% of Level 1, 34.2% of Level 2, and 0.6% of Level 3 evaluations, with penalties attributed to rejection of sound data, failure to carry recognized limitations into conclusions, and wide variation in hypothesis production. Cross-track rankings by
token and code use are substantially more stable than rankings by evidence judgment, suggesting greater consistency in computational effort than in evidence-based
reasoning. These profiles identify opportunities for supervised scientific assistance, but current
agents do not yet demonstrate reliable autonomous analysis or discovery. Code and data: https://huggingface.co/
datasets/discern-bench-anon/discern-
benchmark
Read the full abstract ▾
All authors Nan Huang, Mario Tapia-Pacheco, Kun Zhou, Yiming Huang, Kevin José Barrientos Díaz, Tiffany Amariuta, Jingbo Shang
Artificial Intelligence
Long-Horizon Analog Design Bench: Benchmarking Agents on Hours-Long Analog and Mixed-Signal Circuit Design Tasks
Analog Design Bench Team
Coding
agents now sustain hours-long, tool-driven loops, yet their ability to carry long-horizon analog and mixed-signal
circuits to electrical specification remains unmeasured. We introduce Analog Design Bench, a long-horizon agentic
benchmark of 50 transistor-level design tasks contributed by 17 chip designers.
Agents work with an open-source simulator, while an isolated verifier evaluates the submitted
circuit using specification-based electrical tests. We evaluate 15
agent configurations across 2,250 two-hour attempts and observe full-specification pass rates from 8.0% to 78.0%. Coding-
benchmark performance correlates with analog results but leaves much of the performance spread unexplained. Our failure analysis shows that most unsuccessful submissions have no recorded legality rejection but fail electrical acceptance, identifying electrical closure as the dominant endpoint challenge. We test time,
reasoning effort,
agent harness, and supplied design knowledge as interventions. Longer budgets and higher
reasoning effort improve performance, while general skill documents provide little benefit and sometimes reduce performance. Supplying a task-matched reference
topology, an idealized form of
circuit-IP
retrieval, raises DeepSeek V4 Pro by 18.7 percentage points and mainly accelerates GPT-5.6 Sol.
Read the full abstract ▾
Artificial Intelligence
Unmask the State: When Does State Adaptation Matter for Masked Diffusion Language Models
Injin Kong, Sunghwan Choi and Yohan Jo
Masked
diffusion language models (MDMs) admit flexible generation orders, making the unmasking strategy an inference decision. Existing methods vary in how they prioritize positions, control parallelism, restrict selection regions, revise predictions, or plan future denoising, yet it remains unclear when these choices should change during generation. We study this question through strategy reversals, where an alternative action becomes preferable to a fixed choice. We organize MDM inference into five axes–score, cardinality, region, commitment, and planning–and define adaptation opportunity as the one-step utility advantage of the best candidate action over a validation-selected fixed action. This view shows that adaptation value depends on both the frequency and magnitude of such reversals. Across three MDMs and ten tasks, adaptation opportunities are highly heterogeneous, with some regimes exhibiting concentrated and predictable one-step gains. This motivates selective adaptation: lightweight detectors calibrated on validation
prompts identify high-opportunity states, capturing, for example, 56.9 percent of the candidate-set oracle opportunity by adapting only the top 10 percent of states on LLaDA-8B constrained JSON filling. Our transition-level results suggest that state adaptation is most useful when applied selectively rather than uniformly.
Read the full abstract ▾
Artificial Intelligence
QuPID: Quantum Parameter-Efficient Input-Dependent Retrieval Adaptation for Medical RAG
Hyojun Ahn, Emily Jimin Roh, Soohyun Park et al. (+3 more)
Fidelity-based
quantum retrieval ranks candidates by the fidelity between query and archive states. Applying a shared input-independent
unitary after fixed state encoding leaves that fidelity unchanged, so training the
circuit cannot alter the ranking.
Quantum parameter-efficient input-dependent
retrieval adaptation (QuPID) repairs this by making the
circuit input-dependent through data re-uploading and by comparing measurement readouts, vectors of local Pauli expectations, rather than states. The result is a small readout for adapting frozen image features to a local archive with limited data: training simulates the
circuit classically, and inference runs on a GPU with fixed learned parameters. We characterize the class as a structured factorization of input-modulated quadratic feature maps, bound the frequency support of its re-uploading channel, and give a parameter-count generalization bound that motivates its small budget. Under a shared frozen backbone and a label-free protocol, QuPID's 60 parameters give higher precision-at-5 (P@5) on ChestX-ray14 and MURA than frozen medical encoders, and than adapters and low-rank adaptation (LoRA) with up to 5.25 million trainable parameters. On ChestX-ray14, the P@5 gain over the frozen encoder is +0.116, the lead over retuned adapters is widest at 512 adaptation examples (+0.040), and the full-budget margin over an equally compact classical rotation-plane head is +0.023 with a 95% interval excluding zero. Medical imaging is the primary testbed; the pattern recurs on two non-medical
benchmarks, in report generation, and under simulated gate noise and finite-shot readout.
Read the full abstract ▾
All authors Hyojun Ahn, Emily Jimin Roh, Soohyun Park, Walid Saad, Hyung-Chul Lee, Joongheon Kim
Artificial Intelligence · Cross-listed from cs.LG
KoopCell: Koopman-Based Generative Model for Learning Single-Cell Dynamics from Distribution Snapshots
Wanfeng Lu, Yutong Zhang, Keyi Zhou et al. (+3 more)
Learning population dynamics from temporally sparse, unpaired distribution snapshots is a fundamental challenge in developmental biology. Recent approaches based on neural differential equations and flow matching can interpolate between observed population snapshots, but may struggle to extrapolate beyond the training horizon and often lack an explicit mechanism for modeling developmental branching. We propose KoopCell, a unified generative framework based on Koopman-Mori-Zwanzig theory that jointly learns representations and predictive linear latent dynamics. Theoretically, using the weak continuity equation, we derive a closed-form least-squares
estimator for the Koopman generator from distribution snapshots and establish convergence guarantees under suitable assumptions. To model branching dynamics, we further develop KoopCell-M, which incorporates non-Markovian
memory into the latent Koopman dynamics through a Markovian
embedding. Experiments on synthetic systems and three scRNA-seq
datasets demonstrate the ability of our framework to recover Koopman spectra, model branching through
memory, and scale to predicting high-dimensional gene expression distributions, achieving state-of-the-art performance among the evaluated methods.
Read the full abstract ▾
All authors Wanfeng Lu, Yutong Zhang, Keyi Zhou, Chenxin Ge, Wei Lin, Qunxi Zhu
Artificial Intelligence · Cross-listed from cs.CL
CHI: A Composite Hallucination Index Unifying Entity, Relation, and Quantity Dimensions for Summarization Evaluation
Praveenkumar Katwe, Rakesh Chandra Balabantaray and Kali Prasad Vittala
Faithfulness evaluation of abstractive summaries remains an open challenge, with existing metrics addressing only isolated
hallucination types: factual entity errors, relational inconsistencies, or numerical fabrications, without capturing their co-occurrence or interaction. We introduce CHI (Composite
Hallucination Index), the first unified
hallucination metric that decomposes faithfulness errors into three orthogonal dimensions: entity
hallucination (EHI), relation
hallucination (RHI*), and quantity
hallucination (QHI). Each dimension employs a shared softmax-normalized architecture over Venn diagram-derived factors representing extractiveness, positive
hallucination, over-focus, negative
hallucination, and lost focus. The novel QHI component introduces tolerance-aware numerical matching with exact, epsilon, derived, and temporal comparison modes. We fuse the three dimensions via harmonic mean to produce a single composite score that penalizes weakness in any dimension. We validate CHI on 800 source articles spanning four domains (news, medical, legal, financial) with summaries from five generation systems. Empirical results demonstrate that: (i) the three dimensions are statistically orthogonal (mean rho = 0.148), confirming they capture distinct error types; (ii) CHI achieves the highest system-level correlation with human judgments (rho = 0.66, p = 0.006) on SummEval, outperforming ROUGE (rho = 0.53), EHI (rho = 0.58), and all individual components; and (iii) ablation studies confirm that all three dimensions contribute unique variance, with the full composite outperforming any individual component while providing decomposable error diagnostics unavailable from single-score baselines. CHI provides practitioners with a decomposable, interpretable, and efficient faithfulness metric suitable for both offline evaluation and online monitoring of summarization systems.
Read the full abstract ▾
Artificial Intelligence
Naturalness-guided Manifold Flow Matching for Sign Language Production
Jiayi He, Shengeng Tang, Sisi You et al. (+3 more)
Sign
Language Production (SLP) aims to generate sign motions from text. Conditional Flow Matching methods have achieved strong performance in SLP by constructing conditional paths that transform a source distribution into a target distribution. However, existing methods construct these paths via linear interpolation, whereas the rotational geometry of human joints confines valid joint rotations to a
manifold embedded in Euclidean space. Consequently, linear interpolation between two sign motions leaves this
manifold and ignores the motion distribution on it. In this paper, we revisit SLP from the perspective of
manifold transport and propose a Naturalness-guided
Manifold Flow Matching framework, termed SignNMFlow, which constructs conditional paths directly on the motion
manifold by jointly considering
geometric efficiency and the motion distribution. Specifically, we exploit the intrinsic geometry of the
manifold and introduce a motion naturalness measure to characterize the motion distribution. By minimizing the kinetic energy under this measure, we learn a naturalness-guided interpolation that couples a closed-form geodesic, which provides geometrically efficient transport, with a learnable deviation that incorporates the motion distribution, thereby significantly improving the fidelity of generated sign motions. Extensive qualitative and quantitative evaluations demonstrate the effectiveness of this work.
Read the full abstract ▾
All authors Jiayi He, Shengeng Tang, Sisi You, Yanbin Hao, Lechao Cheng, Richang Hong
Artificial Intelligence · Cross-listed from cs.LG
Beyond Conservatism: Recoverability-Conditioned Exploration for Model-Based Imitation Learning
Xuanlin Chen, Ziyue Wang, Xunlan Zhou et al. (+3 more)
Model-based imitation learning (MBIL) improves real-environment interaction efficiency by optimizing policies on imagined rollouts from a learned world model. However, the gap between model-induced and real-environment occupancies makes policy learning sensitive to model error. Conservative MBIL mitigates model exploitation during policy optimization, but when real-environment interactions are collected by the same conservative policy, uncertain regions around the expert distribution remain insufficiently sampled. Generic uncertainty-driven exploration, on the other hand, may allocate interaction to novel but task-irrelevant dynamics. We propose REcoverability-CONditioned Exploration for Model-Based Imitation Learning (RECON). RECON separates conservative policy learning from active data collection by maintaining a main policy for task execution and an explorer for real-environment interaction. The explorer is optimized based on epistemic uncertainty conditioned on recoverability estimated from multi-step main-policy imagination, focusing data collection on unknown states from which the main policy can still return toward expert behavior. Experiments on locomotion,
navigation and manipulation show consistent gains in interaction efficiency, imitation performance, and robustness, indicating that RECON directs real-environment interaction toward recovery regions around the expert distribution that are underexplored by prior methods, and thereby learns a world model better suited for imitation.
Read the full abstract ▾
All authors Xuanlin Chen, Ziyue Wang, Xunlan Zhou, Yuan-yih Shang, Qiang Wu, Shenghua Wan
Artificial Intelligence · Cross-listed from cs.CL
Does Learning to Predict the World Help Agents Act? Auditing World-Model Post-Training
Xinyu Che, Hang Yan, Yanchen Liu et al. (+5 more)
Predicting how an environment will change before acting is a natural route to better decision making for
agents. Recent post-training methods therefore require
agents to predict the next observation and turn that prediction into a reward or a direct supervision signal, which is called world model. Existing next-observation training methods help the
agent to learn the environmental content. However, they additionally involve an optimization process, which may introduce several effects other than learning to predict the world. Consequently, where the performance gain comes from during the training process remains an open question. We answer this research question through replacing true next-observation targets with in-distribution mismatched observations during the training process. Across two interactive text environments, mismatched targets lower prediction accuracy by 15.3-61.6% relative to ground-truth targets, yet retain substantial task gains over the base model. Compared with the base model, trained models consider more candidate actions and exhibit less looping. We also introduce a setting that replaces prediction-based rewards with independent random signals. This training expands task coverage (pass@64) even when the reward carries no environment information. We also generalize this finding to VisualWebArena, where random-reward training raises pass@64 by 14.3% relative to the base model, without observation-matching rewards or an external
multimodal teacher for reward construction.
Read the full abstract ▾
All authors Xinyu Che, Hang Yan, Yanchen Liu, Haochen Liu, Ruifeng Li, Anran Shi, Heng Wang, Jun Liu
Artificial Intelligence
ANTMAN: Adaptive Need Tracking for Multi-Agent Navigation in Large Information Spaces
Jerry Wang, Haibo Jin, Xiaopeng Yuan et al. (+2 more)
Information-seeking
agents increasingly operate over information spaces that are too large to process exhaustively. Yet many multi-
agent systems organize computation around static partitions of the available space, causing coordination to grow with how information is segmented rather than with what the query still requires. We introduce ANTMAN, an adaptive coordination framework that treats evolving unresolved information needs as the unit of runtime coordination. ANTMAN maintains a revisable Need Graph that tracks unresolved requirements, accumulated evidence, prior attempts, and search progress, and uses this state to control worker selection, routing, and task-local recovery as new evidence is discovered. By separating the coordination policy from substrate-specific search interfaces, the same need-conditioned mechanism can operate across different information spaces. Experiments across multi-document question answering, controlled long-context scaling, and realistic structured
navigation show that ANTMAN remains effective across settings, including when execution is delegated to substantially smaller worker models. Under a 16x increase in searchable context, ANTMAN increases active coordination by only 1.23x, compared with more than 15x for partition-driven baselines, while preserving strong answer quality.
Read the full abstract ▾
All authors Jerry Wang, Haibo Jin, Xiaopeng Yuan, Peng Kuang, Haohan Wang
Artificial Intelligence
Thanh-Dat Truong, Sankalp Pandey, Hugh Churchill et al. (+3 more)
Recent advances in Large
Language Models (LLMs) have enabled agentic systems capable of solving complex tasks through multi-turn planning, tool use, verification, and
memory updates. However, learning agentic systems remains difficult due to two fundamental challenges, i.e., (1) long-horizon credit assignment, where supervision is available only at the final outcome, and (2) imbalanced data distributions, where dominant data patterns bias optimization and weaken adaptation to rare but informative
reasoning behaviors. In this paper, we propose Fair Multi-Level Preference Optimization (Fair-MPO or $\Phi$-MPO), a new preference optimization framework for agentic learning. We first show that Multi-Level Preference Optimization provides a principled and more computationally efficient framework for long-horizon
reasoning. Then, we introduce a Fair Multi-Level Objective that addresses imbalance in agentic learning. We provide a comprehensive theoretical analysis demonstrating that our approach addresses both long-horizon
reasoning and data imbalance. Our experiments on agentic
reasoning benchmarks demonstrate that our approach achieves State-of-the-Art (SOTA) performance.
Read the full abstract ▾
All authors Thanh-Dat Truong, Sankalp Pandey, Hugh Churchill, Jackson Cothren, Marios Savvides, Khoa Luu
Artificial Intelligence · Cross-listed from cs.DB
Robust Hierarchical Structures for Agentic Document Analysis
Ruiying Ma, Yiming Lin and Aditya G. Parameswaran
Large
Language Models (LLMs) enable us to better understand text documents, including PDFs and Word documents. However, LLMs, as well as more modern LLM
agents, i.e., those with tool-calling abilities, typically treat such documents as plain text, ignoring the fact that they are often organized hierarchically into sections and subsections. Extracting this structure, while difficult, can improve efficiency and effectiveness for
agents (and humans)—since only sections relevant to a given task need to be processed. Unfortunately, prior work on structure extraction provides no formal guarantees on how well the inferred structure matches the true one. Instead, we target a robust and compact variant that is feasible to infer and useful in practice. Robustness ensures that the text under each subsection header is a superset of the text under the same header in the true structure. Compactness seeks to minimize this superset, reducing agentic cost (or human cognitive load). We propose SHED, a two-stage workflow for inferring a robust and compact structure. The first stage is pluggable with an infinite family of approaches, each guaranteeing robustness for a specific document class. We theoretically characterize the document space using these classes and their hierarchical relationships. Empirically, SHED improves F-1 scores (measuring the robustness–compactness trade-off) by 13%–68% over non-LLM baselines and 9%–15% over expensive LLM-based approaches. Finally, we show how SHED-inferred structures are valuable for agentic document analysis:
agents using SHED outperform baselines, achieving 3%–23% higher accuracy while being up to 10x cheaper.
Read the full abstract ▾
Artificial Intelligence
Kecheng Liang, Haoyang Liu, Zexin Chen et al. (+5 more)
A key challenge in physics diagram understanding is correctly associating visual information with the physical entities, relations, and conditions it describes. Even when a value, symbol, or other local element is accurately recognized, assigning it to the wrong entity or scope can distort the underlying physical premise and lead to incorrect
reasoning. To systematically study this challenge, we introduce PhysAlign, a
benchmark designed to assess whether
multimodal models correctly associate information recognized from physics diagrams with its intended physical role. By disentangling visual recognition from physical-role assignment through localized probes and controlled variants, PhysAlign isolates correspondence errors from recognition failures. It contains 3,341 human-validated probes spanning 986 physics problems, enabling systematic evaluation of visual recognition and physical-role correspondence at scale. We further introduce five complementary evaluation metrics, including CAcc, GAcc, and JAcc, which provide a comprehensive assessment of models' ability to recognize diagram content, establish correct physical correspondences, and solve the underlying physics problem. Across our evaluated
multimodal models, PhysAlign reveals a consistent gap between local visual recognition and physical-role grounding. Even when the queried content is correctly recognized, the conditional correspondence error rate remains 13.8% for GPT-6-Astra and rises to about 50.6% for InternVL3.5-8B. These findings indicate that strong perception alone does not ensure reliable physical interpretation, exposing a distinct grounding bottleneck that is largely hidden by answer-level accuracy and highlighting the need for future models to better align recognized visual evidence with its physical meaning.
Read the full abstract ▾
All authors Kecheng Liang, Haoyang Liu, Zexin Chen, Zirong Liu, Weixing Chen, Qiufeng Wang, Yang Liu, Liang Lin
Artificial Intelligence · Cross-listed from cs.LG
ZeroGAR: Benchmarking the Adversarial Robustness of Zero-Shot Graph Models
Zhongjian Zhang, Xiao Wang, Busheng Zhang et al. (+5 more)
Zero-shot graph models (ZGMs), which learn transferable knowledge from source graphs and directly apply to unseen target graphs without any adaptation, have achieved promising performance and attracted considerable
attention. Despite their proliferation, existing ZGMs are predominantly evaluated on clean graphs, while existing graph robustness
benchmarks mainly focus on supervised settings, leaving a fundamental question largely unexplored: How robust are ZGMs when their unseen target graphs are exposed to
adversarial manipulation? In this paper, we answer this question by proposing ZeroGAR, the first systematic
benchmark for evaluating the
adversarial robustness of ZGMs. ZeroGAR evaluates 13 representative ZGMs from 3 different paradigms on 8 graph
datasets across 4 domains, covering both in-domain and cross-domain transfer under structural, textual, and node injection attacks with multiple perturbation budgets. It further investigates whether existing graph defenses remain effective in the zero-shot setting. Extensive experiments reveal that strong clean zero-shot performance does not guarantee
adversarial robustness, with three key findings: (1) Vulnerability patterns are related to model prediction mechanisms: GNN-based methods are particularly vulnerable to structural and node injection attacks, whereas LLM-based methods are more vulnerable to textual attacks; (2) Stronger LLM backbones introduce a structure-text robustness trade-off; (3) Existing graph defense methods do not consistently improve zero-shot robustness and may compromise clean performance. We hope that ZeroGAR will facilitate rapid, equitable evaluation and inspire further innovative research in ZGM security.
Read the full abstract ▾
All authors Zhongjian Zhang, Xiao Wang, Busheng Zhang, Bo Yan, Xingtong Yu, Yue Gao, Jia Li, Chuan Shi
Artificial Intelligence · Cross-listed from cs.LG
BITS: Rethinking Fair and Comprehensive Evaluation for Irregular Time Series Forecasting
Kangjia Yan, Linfeng Wang, Tianen Shen et al. (+7 more)
Despite recent progress in irregular time series
forecasting, the field still lacks a unified
benchmark for fair and comprehensive evaluation. Existing evaluations are often conducted on a limited set of
datasets with inconsistent experimental protocols and predominantly error-based metrics, rendering it difficult to compare and assess methods fairly and comprehensively across diverse settings. To eliminate these limitations and accelerate progress, we propose BITS, a standardized, reproducible, and extensible
benchmark for advancing research on irregular time series
forecasting. BITS covers eleven
datasets from nine domains with diverse irregularity characteristics, and it characterizes the
datasets according to their missing rate, missing pattern complexity, sampling irregularity, and skewness. Further, it offers a unified pipeline for data preprocessing, model integration and evaluation, and reporting. It accommodates regular and irregular time series
forecasting methods, including time series foundation models, under consistent settings, incorporating both error-based and non-error-based evaluation metrics. Findings include that method performance varies substantially across irregularity characteristics, with no single modeling strategy consistently dominating. We also find that using error-based or non-error-based metrics can yield different model rankings, highlighting the need for multi-dimensional evaluation. The code can be found at https://anonymous.4open.science/r/BITS-8F2E/.
Read the full abstract ▾
All authors Kangjia Yan, Linfeng Wang, Tianen Shen, Xiangfei Qiu, Ruitong Zhang, Hao Miao, Jilin Hu, Chenjuan Guo, Bin Yang, Christian S. Jensen