Artificial Intelligence · Cross-listed from cs.RO
GLST: Global-Local Spatio-Temporal Memory Decomposition Framework for Online Chained-Goal Navigation
Yousef Liang, Ryan Ardalan, Yarden Goraly et al. (+6 more)
Autonomous navigation in real-world environments often requires long-horizon operation under partial observability and evolving goals. Existing mapless navigation approaches predominantly assume fixed goals throughout an episode, limiting applicability to practical scenarios where goals change online during execution. We study online chained-goal navigation, where goals evolve through existing-goal redirection or new-goal emergence without explicit transition signals. Under this setting, we find navigation may require either continued exploration or strategic revisitation, creating conflicting
memory requirements. We propose Global-Local Spatio-Temporal Belief Learning (GLST), a
memory-decomposition framework that separates latent beliefs into global-local spatial and long-short temporal representations for long-horizon
reasoning. We further introduce goal-conditioned rewards to balance exploration and exploitation under evolving goals. Extensive experiments in simulation across diverse terrains demonstrate improved chained-goal navigation performance over baselines. Real-
robot experiments using the Spot quadruped further validate our
memory decomposition framework.
Read the full abstract ▾
All authors Yousef Liang, Ryan Ardalan, Yarden Goraly, Chenhao Zhu, Yue Yin, Jonas Frey, Claire Tomlin, Marco Pavone, Shankar Sastry
Artificial Intelligence · Cross-listed from cs.CV
CineMem: Attention-Aligned Fine-grained Memory Retrieval for Multi-Shot Long Video Generation
Xiaoxue Wu, Xinyu Gao, Fang Yu et al. (+2 more)
Autoregressive
video generation relies on
memory retrieval to model long-range dependencies. This capability becomes particularly critical for multi-shot generation, where models must reconcile substantial visual changes across shot transitions with persistent semantic continuity. Existing methods tend to perform
memory retrieval at the frame level and share the same historical context across all query
tokens, overlooking the heterogeneous information needs and leading to redundant
retrieval and inefficient use of historical
memory. To bridge this gap, we introduce CineMem, an
attention-aligned fine-grained
memory retrieval framework that combines online
memory organization with query-adaptive
retrieval for autoregressive multi-shot
video generation. Specifically, guided by intrinsic
memory-access patterns revealed through
attention analysis, we propose an online-clustered KV cache as a
memory organization strategy over all historical
tokens, enabling semantic-aware
memory grouping and more precise
retrieval. In addition, we introduce a query-level
retrieval mechanism that provides broader access to historical information under a fixed
retrieval budget, thereby improving long-term
memory utilization. Together, these designs enable the multi-shot
video generator to preserve long-range consistency across shot transitions, particularly in revisitation scenarios, while approximating full-
memory access under a constrained
memory budget. To facilitate evaluation, we further introduce a suite of task-specific, VLM-assisted metrics for faithful inter-shot consistency assessment. Extensive experiments demonstrate the effectiveness of our framework.
Read the full abstract ▾
All authors Xiaoxue Wu, Xinyu Gao, Fang Yu, Yu Qiao, Xinyuan Chen
Artificial Intelligence · Cross-listed from cs.CV
LinMorph: Learning to Morph Weights for Efficient Video DiT Linearization
Lingyi Long, Congyue Zhao, Xiaoou Li et al. (+2 more)
Post-hoc linearization is an attractive way to accelerate pretrained
Video Diffusion Transformers (
Video DiTs), yet existing methods largely treat the quality loss induced by
attention replacement as a recovery problem. We show that a more fundamental bottleneck lies upstream: initialization. Random initialization discards pretrained knowledge, whereas manual weight reuse transfers it through rigid and incomplete source–target correspondences. We introduce LinMorph, which recasts
Video DiT linearization as a learnable softmax-to-linear weight transformation problem. Instead of copying predefined counterparts, LinMorph learns a cross-parameter morphism that lets each target Gated DeltaNet (GDN) parameter draw jointly from pretrained query, key, value, and output projections, including for parameters with no direct softmax counterpart. A single morphism is shared across layers and learned through local teacher
alignment, while producing layer-specific initializations from their respective pretrained weights. The resulting weights are then materialized as independent GDN modules for progressive
distillation and joint recovery.Since the morphism operates in weight space rather than
token space, it can be learned on short, low-resolution clips and directly reused at larger
video scales. Across pretrained
Video DiTs, LinMorph enables markedly faster recovery and better final generation quality than random initialization and heuristic weight reuse. Results show that efficient linearization depends not only on how a converted model is recovered, but critically on how pretrained knowledge is transferred before recovery. Project page: https://lonelyling.github.io/LinMorph.
Read the full abstract ▾
All authors Lingyi Long, Congyue Zhao, Xiaoou Li, Yanmin Qian, Tao Huang
Artificial Intelligence · Cross-listed from cs.SE
SpecCloak: Probing Harmful Code Generation through Engineering Specifications
Wenbin Chen, Jiexin Wang, Guanting Wu et al. (+2 more)
The code generation capabilities of large
language models (LLMs) can equally be misused to implement harmful functionality, underscoring the need to understand their safety behavior in response to programming requests. While request phrasing is known to influence refusal, it remains unclear whether the form in which coding task requirements are stated affects the refusal process for malicious code generation. We introduce PseudoSpec, a template that organizes functional requirements as engineering specifications, and investigate its effects through controlled comparisons, representation analysis, and activation interventions. Across five open-weight models, removing explicit risk keywords only partially reduces refusal, whereas specification prompting substantially lowers refusal even when these keywords are retained. This reduction accompanies consistent shifts toward code-output-related representations, and interventions along condition-difference directions alter refusal behavior. Output analysis further distinguishes code-related responses from generation degeneration. Building on these findings, we propose SpecCloak, a black-box red-teaming framework combining PseudoSpec with discriminator-guided optimization, achieving an average execution-verified attack success rate of 28.91% across eight target models on RedCode-Gen. We also introduce SpecReveal, a defense prototype combining natural-
language intent reconstruction with dual-view detection without target-model fine-tuning. It achieves a 96.88% attack detection rate in a separate
prompt-detection evaluation, with false positives assessed on benign programming and specification-style inputs. These results connect specification-induced response differences to practical security consequences and identify intent reconstruction as a complementary detection approach.
Read the full abstract ▾
All authors Wenbin Chen, Jiexin Wang, Guanting Wu, Jiaxi Liang, Yi Cai
Artificial Intelligence · Cross-listed from cs.CV
DriveWLA: Unifying Future Dynamics and Scene Semantics through World-Language-Action Modeling for Autonomous Driving
Shuzheng Wang, Haoxuan Li, Shengzhen Li et al. (+5 more)
Vision-
Language Models (VLMs) and world models have demonstrated strong capabilities in semantic understanding and future scene
evolution for autonomous driving. However, effectively integrating both capabilities for trajectory generation remains challenging. In this work, we propose DriveWLA, a unified World-
Language-Action model based on Mixture-of-
Transformers (MoT) that jointly models semantic understanding, future scene dynamics, and trajectory generation. DriveWLA consists of three experts: a World Expert for future prediction, a
Language Expert for semantic understanding, and an Action Expert for trajectory generation. We further introduce Cross-Expert
Attention, which enables controlled information flow across experts while aligning semantic and dynamic representations. To improve trajectory generation beyond imitation learning, we apply Flow-GRPO reinforcement learning post-training to refine the Action Expert. Extensive experiments show that DriveWLA achieves superior performance in both open-loop and closed-loop evaluation. DriveWLA achieves 92.3 PDMS on NAVSIM v1 and 90.1 EPDMS on NAVSIM v2, demonstrating strong performance among VLM-based and world-model-based planners. Without specific fine-tuning, DriveWLA further demonstrates strong zero-shot generalization to HUGSIM, nuScenes, and Bench2Drive. These results validate the effectiveness of our unified World-
Language-Action modeling, establishing DriveWLA as a generalizable framework for autonomous driving.
Read the full abstract ▾
All authors Shuzheng Wang, Haoxuan Li, Shengzhen Li, Chenxi Wu, Yajing Sun, Lu Hou, Wenyu Liu, Xinggang Wang
Artificial Intelligence · Cross-listed from cs.CV
Visual Dependence Does Not Imply Answerability: A Systematic Analysis of Answerability Signals in Frozen InstructBLIP
Minjun Kim, Hyunmin Song and Changwon Lim
Visual question answering models often produce plausible responses even when the available visual evidence is insufficient to support a valid answer. Although prior work has introduced
benchmarks and abstention mechanisms for unanswerable visual questions, it remains unclear which signals derived from a frozen
vision-
language model reliably indicate answerability on unseen image-question pairs. We systematically evaluate answerability signals on a stratified 10K subset of UNK-VQA using frozen InstructBLIP, covering predictive uncertainty, cross-modal
alignment, visual grounding, visual dependence, internal representations, and learned probes. Answerability discrimination remains weak, with test AUROC largely confined to 0.50-0.54. CLIP
alignment yields the highest point estimate but does not significantly outperform generation confidence under paired bootstrap analysis. Targeted removal of question-relevant visual evidence causes substantially larger changes in answer confidence than same-size random masking, yet intervention sensitivity provides essentially no answerability discrimination. Increasing probe capacity, combining multiple signals, and adding reason-aware supervision do not provide robust improvements, while post-freeze cross-validation yields similarly weak discrimination. Selective-answering analysis shows limited usable coverage under low-risk operating criteria. Failure-mode analysis shows that the relative utility of individual signals varies across perturbation mechanisms. Taken together, these results show that, on our UNK-VQA subset with frozen InstructBLIP, visual dependence, uncertainty, semantic
alignment, and static representations are not robust standalone proxies for answerability. Our findings emphasize the need to distinguish a model's reliance on visual evidence from whether that evidence is sufficient to support an answer.
Read the full abstract ▾
Artificial Intelligence
PaceReasoner: Label-Free Learning to Answer Briefly from Long Reasoning
Changguo Jia, Zhiyou Xiao, Xinbei Ma et al. (+5 more)
Extended chain-of-thought
reasoning improves problem-solving performance but increases inference cost and latency, motivating more accurate answers with shorter
reasoning. We introduce PaceReasoner, a label-free test-time reinforcement learning framework that uses a model's LONG responses to improve its SHORT-mode accuracy. Both modes share a trainable policy, creating a need for reliable supervision, sufficient SHORT training samples, and continued LONG responses to guide learning. PaceReasoner combines all-response and LONG-only answer consensus through progressive, confidence-weighted supervision. A behavior reward gradually shifts from encouraging SHORT exploration to rewarding agreement with LONG answers, while a ratio controller regulates the balance between modes. Across three model backbones and four
benchmarks, PaceReasoner improves average SHORT accuracy over the baselines by 7.4–13.5 percentage points, with fewer than 25
reasoning tokens per response on average. Ablations support the key components, and cross-
dataset results demonstrate transfer beyond the adaptation set.
Read the full abstract ▾
All authors Changguo Jia, Zhiyou Xiao, Xinbei Ma, Geng Chen, Weiming Zhang, Yiming Xu, Tianqi Zhao, Minghui Zhou
Artificial Intelligence · Cross-listed from cs.LG
Evaluating the Causal Faithfulness of SHAP as a Candidate Attribution Method for SAE Features
Samuel Xie and Noura Al-Moubayed
Sparse autoencoders decompose
language model representations into interpretable features, but feature interpretation in this literature remains predominantly correlational. Attribution methods offer more direct estimates of feature influence, yet their faithfulness to actual causal effects has not been systematically evaluated in the SAE setting. We apply SHAP-based attribution to SAE latent features and evaluate it against direct causal intervention, using sentiment classification on SST-2 with GPT-2 Small and a 32,768-feature layer-7 SAE as a testbed. We compare SHAP to Integrated Gradients, Gradient$\times$Input, and probe-weight baselines across linear and MLP probe arms, scoring each against feature ablation and activation steering measured through the
language model's own logit difference. Most faithfulness associations we measure do not survive multiple-comparison correction, and attribution methods that agree strongly with one another ($\rho = 0.42$–$0.80$) agree poorly with measured causal effects, indicating that inter-method agreement is not evidence of causal faithfulness. Two effects hold under control. First, local per-example attribution retains a directional advantage over global averaged attribution, while its apparent importance advantage is explained by feature activity rather than superior causal ranking. Second, under known-ground-truth minimal pairs, attribution recovers inserted sentiment signal well above chance (AUROC $0.792$) but fails to localize deletions (AUROC $0.520$), suggesting that these methods detect the presence of a feature's influence more reliably than its absence. We argue that importance faithfulness and directional faithfulness are separable properties requiring separate evaluation, and that attribution agreement should not substitute for causal validation.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Interaction-as-a-Judge
Wayne Chi, I-Hung Hsu, Li Chen et al. (+6 more)
Coding
agents are tasked with producing sophisticated interactive applications such as websites, desktop applications, and games. While humans assess these applications interactively, current automated LLM evaluations primarily focus on code analysis and unit tests. This mismatch can cause LLM judges to miss failures that are immediately apparent to human users. We study Interaction-as-a-Judge, an evaluation paradigm where computer-use
agents (CUAs) directly interact with an application. We demonstrate that Interaction-as-a-Judge can improve both evaluation and generation of interactive applications. To this end, we collect and release two complementary
datasets: $6,858$ pairs of applications with human preferences and corresponding
agent traces, and $222$ applications with human-verified requirement labels. Relative to standard non-interactive approaches, we find that Interaction-as-a-Judge improves evaluation in terms of predicting human preferences ($65.6\% \to 69.1\%$) and requirement verification ($77.0\% \to 81.6\%$). We also observe scaling benefits: across both preference
alignment and requirement verification, higher-quality interactions improve performance. Finally, through both inference- and training-time experiments, we demonstrate that Interaction-as-a-Judge can also improve the quality of generated artifacts. At inference time, Interaction-as-a-Judge repairs $75.5\%$ of broken requirements within five attempts, compared to $55.4\%$ for code-only self-feedback. At training time, using Interaction-as-a-Judge as a reward signal improves win-rate over the base model by $6.7$pp while reducing both generated code length by $35.7\%$ and build failure rates by $7.2$pp compared to a code-only reward signal.
Read the full abstract ▾
All authors Wayne Chi, I-Hung Hsu, Li Chen, Wei-Lin Chiang, Ion Stoica, Anastasios Nikolas Angelopoulos, Ameet Talwalkar, Chris Donahue, Cho-Jui Hsieh
Artificial Intelligence
Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards
Keru Chen, Sen Lin, Yingbin Liang et al. (+2 more)
Effective collaboration among large
language model (LLM)
agents in multi-
agent systems requires sharing relevant evidence, requesting missing information, and incorporating teammates' contributions into decisions. Training these behaviors is challenging because final task outcomes provide limited feedback on the value of individual messages. We introduce CoSIR (Collaboration via Surprisal-based Information Rewards), a post-training approach that uses a simple, verifiable environment to teach collaboration behaviors that generalize to more complex, unseen tasks. We propose a
Bayesian evaluator that scores shared information by its reduction in team surprisal, defined as the sum of the negative log probabilities assigned to the correct option given each
agent's available evidence. Requests are scored by the potential reduction in the same quantity if the requested information were truthfully provided. Separate rewards encourage
agents to use the received information in their private choices and calculations. We post-train Qwen3-4B, Qwen3-8B, and Gemma-3-4B-it through self-play and evaluate transfer to seven unseen natural-
language collaboration tasks. CoSIR achieves the highest aggregate score, while its performance in general tasks remains competitive. Behavioral analyses show increased disclosure of private evidence and fewer judged failures to use teammates' input.
Read the full abstract ▾
All authors Keru Chen, Sen Lin, Yingbin Liang, Nathaniel D. Bastian, Shaofeng Zou
Artificial Intelligence · Cross-listed from cs.CL
Cross-Lingual Transfer of Evil Vectors Is Directionally Asymmetric
Juann Kim and Minku Kang
Persona vectors are linear directions in the residual-stream activations of LLMs that encode behavioral traits and enable fine-grained control over model behavior. Among them, the evil persona vector is particularly relevant to understanding harmful behavior, jailbreak vulnerabilities, and the unlearning of harmful knowledge. Yet whether this vector reflects a genuinely cross-lingual concept, rather than an artifact of its extraction
language, remains untested. We conduct a systematic evil-vector transfer study across two high-resource
languages (English, Spanish) and two low-resource
languages (Swahili, Telugu) on four open-source instruction-tuned LLMs, and find that transfer is directionally asymmetric. Evil vectors are harder to obtain reliably from low-resource
languages, and the resulting vectors are largely ineffective at steering, even within their own
language. In contrast, vectors extracted from high-resource
languages transfer effectively to low-resource inputs while largely preserving general knowledge, as measured by MMLU.
Geometric analysis further supports this pattern: evil vectors from the two high-resource
languages are closely aligned, whereas low-resource vectors align only weakly with this shared direction. These results suggest that a shared cross-lingual evil direction exists and, once reliably extracted from a high-resource
language, can be reused to steer behavior in low-resource
languages without additional per-
language data collection.
Read the full abstract ▾
Artificial Intelligence
ECMR: Evidence-Aware Deliberation Control for Multi-Agent LLM Reasoning
Yuan Liu, Haoyue Peng, Fanliang Hu et al. (+2 more)
Multi-
agent Large
Language Model (LLM) systems typically commit to a deliberation protocol before inference. A fixed protocol cannot allocate computation differently to resolved and unresolved instances, and it may mistake repeated support for independent confirmation. We study this failure as semantic false consensus: a confident group answer supported by narrow, duplicated, or correlated rationales. We introduce ECMR, an evidence-aware controller over frozen role-conditioned LLMs. At each step, ECMR summarizes answer concentration, support diversity and independence, contradictions, role behavior, verification history, and remaining budget, then selects whether to propose, challenge, verify, arbitrate, diversify, correct a role, or stop. We formulate this decision as finite-horizon budgeted control and derive sufficient conditions under which state conditioning, risk-triggered verification, and role correction improve cost-adjusted utility. Across a declared held-out evaluation of 9,000 records from MedQA, CaseHOLD, and FinQA, ECMR-Full increases pooled accuracy from 62.1% to 84.9%, reduces false-consensus rate from 40.7% to 17.0%, and raises evidence-support rate from 58.0% to 88.7%. State shuffling loses 11.7 accuracy points with only 0.7 fewer rounds, while harder traces shift from early stopping toward arbitration. ECMR-Full uses the largest
token budget, yielding the strongest absolute reliability rather than uniformly superior
token efficiency.
Read the full abstract ▾
All authors Yuan Liu, Haoyue Peng, Fanliang Hu, Lei Huang, Wenli Cai
Artificial Intelligence · Cross-listed from cs.LG
RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions
Rishabh Ranjan, Vignesh Kothapalli, Harshvardhan Agarwal et al. (+5 more)
Foundation models for tables and relational databases are increasingly used for predictive tasks across various scientific and industrial domains. However, adapting existing models to unseen
datasets requires tens of thousands of examples. Here we introduce RT-J, an 86M-parameter relational foundation model (RFM) that makes accurate predictions even with hundreds of in-context examples or fewer. To achieve this, we first assemble the Join, a large collection of 639 web-crawled databases with 6,023 LLM-generated forecasting tasks. We pretrain with long contexts of up to 8,192 database
cells, using in-context examples from a novel lightweight random-walk retriever. Finally, we leverage context tuning and ensembling to scale up test-time compute. On in-context regression tasks from RelBench, RT-J requires 10x fewer examples on average to match strong feature engineering pipelines with LLM-
agent or RDBLearn featurizers and TabICLv2 or LightGBM predictors. Overall, our work pushes the frontier for context-efficient predictions on relational data and highlights the importance of real-world semantics, context engineering and large-scale pretraining for RFMs.
Read the full abstract ▾
All authors Rishabh Ranjan, Vignesh Kothapalli, Harshvardhan Agarwal, Charilaos Kanatsoulis, Roshan Upendra, Tom Palczewski, Carlos Guestrin, Jure Leskovec
Artificial Intelligence · Cross-listed from cs.LG
TStruct: Learning Shared Temporal Structures for Long-Term Time-Series Forecasting
Yangze Li, Zhengyang Zhou, Qihe Huang et al. (+3 more)
Long-term time series forecasting requires temporal dependency modeling that remains stable and generalizable under temporal distribution shifts. However, real-world time series are often noisy and non-stationary, making deep learning models prone to fitting incidental temporal interactions in the training data. A key question is whether there exist temporal relationship structures that recur across different temporal positions and variables. We analyze temporal relationship
matrices on commonly used time series
datasets and find that samples from different temporal positions and different variables exhibit shared dominant temporal relationship structures. This indicates that temporal relationship structures contain patterns that recur stably across samples and variables. To learn such shared structures, we propose Temporal Structure Network (TStruct), a simple yet effective network for directly learning shared and generalizable temporal dependencies. Specifically, TStruct directly parameterizes a set of shared temporal dependency
matrices and generates normalized combination weights using dynamic sample information and static variable priors. These weights represent a soft allocation of each sample-variable pair over shared temporal dependency structures, enabling the model to generate adaptive temporal dependency
matrices under shared structural constraints, rather than fitting incidental interactions in a fully unconstrained manner. Notably, this simple structure-aware design consistently outperforms several strong baselines across all
datasets, achieving superior accuracy while maintaining high computational efficiency.
Read the full abstract ▾
All authors Yangze Li, Zhengyang Zhou, Qihe Huang, Pengkun Wang, Binwu Wang, Yang Wang
Artificial Intelligence
Socratic Skill Learning: Interleaving Teaching and Practice for Transferable Agent Skills
Letian Zhu, Han Li, Yifan Yao et al. (+9 more)
Agent Skills improve LLM
agents by making task knowledge reusable across sessions. Learning transferable Skills requires evidence about a procedure and its conditions of use, which a successful task attempt can leave untested. We introduce Socratic Skill Learning (SSL), which interleaves expert teaching with Student practice. An Educator guides investigation while the Student remains the sole environment operator and Skill author, able to question, test, and adapt advice. To test reuse beyond the learning task, we construct SkillTransferBench, with 35 task families and 408 executable cases, pairing public variation specifications with held-out evaluation in fresh solver sessions. On the main 22-family, 155-case cohort, Fixed-Window SSL reaches 0.801 utility, versus 0.723 without a Skill, 0.761 for adapted Trace2Skill (trajectory
distillation), and 0.751 for adapted SKILL-KD (contrastive
distillation). Across three model pairs on shared families, interleaved guidance outperforms expert authoring and post-hoc teaching. Process analysis links guidance to Student writing and package revision; a case study shows the Student testing and refining a proposed procedure for later reuse. As demand for new lessons falls, the cost of fixed-window review grows. Issue-Guided SSL tracks unresolved questions across Skill revisions to select and focus review, reaching 0.805 utility with 61.7% fewer Educator
tokens and 16.9% fewer total acquisition
tokens than Fixed-Window SSL on the main cohort.
Read the full abstract ▾
All authors Letian Zhu, Han Li, Yifan Yao, Xinping Lei, Xinyu Che, Jiahao Wang, Dianyi Wang, Yucheng Fu, Jiakai Wang, Zhiqi Bai, Wenbo Su, Jiaheng Liu
Artificial Intelligence · Cross-listed from cs.LG
MMTA: Benchmarking Multimodal Temporal Analysis with Time Series, Text, and Vision
Ziyang Zhang, Shenyi Li, Yilin Wang et al. (+4 more)
Time-series analysis is increasingly studied in
multimodal settings, yet most
benchmarks pair temporal signals with text or with visualizations derived from the same signal. This leaves open whether
multimodal large
language models (MLLMs) can integrate time series with real-world visual evidence. We introduce MMTA (
multimodal temporal analysis), a
benchmark of tri-modal tasks with diverse cross-modal complementarity patterns among time series, text, and
vision. MMTA spans nine domains – robotics, autonomous driving, human action, manufacturing, AR assistance, healthcare, recommendation, finance, and
climate – under
video+TS and image+TS layouts, with a 3,524-sample test split covering classification and forecasting. The
benchmark standardizes heterogeneous real-world sources into a shared schema with domain-specific
prompts, provenance documentation, and success-weighted metrics that penalize both wrong answers and failed generations. Evaluating eight open-source and proprietary MLLMs reveals a shared bottleneck: representing time series as text creates very long
token sequences and weakens fine-grained
alignment with visual evidence, especially in long-
video settings. We then introduce TimeOmni-v, a
multimodal LLM extension with a dedicated time-series encoder, temporal positional
alignment, modality-aware
token layout, and a forecasting head. TimeOmni-v improves long-
video classification and time-series inference efficiency, showing that MMTA not only exposes representation and grounding failures but also motivates targeted model design.
Read the full abstract ▾
All authors Ziyang Zhang, Shenyi Li, Yilin Wang, Ziyun Cui, Bowen Zhou, Wen Wu, Chao Zhang
Artificial Intelligence · Cross-listed from cs.IR
FASHION-RQ: Mining Hierarchical Semantic Structure for Automated Evaluation of Fashion Retrieval
Tripti Shukla, Shwetha Somasundaram, Akshat Gupta et al. (+3 more)
Evaluating fashion
retrieval systems remains challenging due to the limited availability of
benchmarks that capture the fine-grained, open-ended queries encountered in real-world search and recommendation. Existing
benchmarks often rely on manual curation, predefined taxonomies, or synthetic
prompts, limiting scalability and semantic coverage. We introduce FASHION-RQ: Mining Hierarchical Semantic Structure for Automated Evaluation of Fashion
Retrieval, an automated framework that derives user-type queries and relevance sets directly from catalog structure. Our key insight is that hierarchical discrete representations learned by Residual Quantized Variational Autoencoders (RQ-VAE) organize products into semantically coherent clusters at progressively finer granularities. We leverage this structure to construct FASHION-RQBENCH, a scalable
benchmark without manual query annotation or predefined taxonomies. We propose four complementary tasks for evaluating ranking, relevance classification, fine-grained sibling discrimination, and paraphrase robustness, alongside query- and attribute-level relevance metrics. Experiments across recent
retrieval models demonstrate that FASHION-RQBENCH differentiates model capabilities across query types and attributes. A human study further shows that
benchmark queries achieve comparable naturalness and higher intent clarity than real user queries, supporting their validity for real-world evaluation. The code and FASHION-RQBENCH
dataset will be publicly released upon acceptance.
Read the full abstract ▾
All authors Tripti Shukla, Shwetha Somasundaram, Akshat Gupta, Rohit Garg, Paul Duff, Ian Anderson
Artificial Intelligence
Learning Adaptive Deliberation Control for Reliable Recognition with Multi-Agent Language Models
Yuan Liu, Haoyue Peng, Fanliang Hu et al. (+2 more)
Multi-
agent deliberation can improve recognition by criticizing and revising candidate answers, but fixed controls cannot respond to changes in disagreement or
reasoning quality. We present an adaptive multi-
agent controller (ADMA) that learns continuous policies for proposal, challenge, arbitration, and coordination while keeping the task model frozen. Shared trace features describe quality, agreement, and progress, while role-specific features support decentralized control during inference. One controller trained on legal and financial
reasoning is transferred without policy updates to four textual and
multimodal benchmarks and two successive task-model versions. With Qwen3.5 held fixed, ADMA-Full improves accuracy by 6.2-14.2 percentage points over Direct and by 2.8-3.0 points over ADMA-Fixed. Paired bootstrap ranges for the Full-Fixed comparison are positive on three tasks. Outcome decompositions show that recoveries exceed introduced errors, while process assessment records fewer false-consensus events and more supported correct outcomes. Transfer to Qwen3.6 remains task dependent, which bounds the generality of the learned policy. Median
token and latency ratios are 14.3-18.7 and 7.8-16.5 relative to Direct. These results show that adaptive deliberation can improve fixed-model recognition when its task-dependent gains justify the additional computation.
Read the full abstract ▾
All authors Yuan Liu, Haoyue Peng, Fanliang Hu, Lei Huang, Wenli Cai
Artificial Intelligence · Cross-listed from cs.CR
ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities?
Fasheng Miao, Zhaoyu Hu, Xiang Li et al. (+5 more)
Large
language model (LLM)
agents are increasingly used for vulnerability analysis, yet existing
benchmarks provide limited coverage of exploitation that depends on interactions among network protocol components. Such vulnerabilities arise from inconsistencies in message interpretation, timing, state, or trust assumptions across participants. We introduce ProtoArena, a black-box, multi-component, dynamic
benchmark with 80 cross-component vulnerability-exploitation environments spanning 11 protocol and application domains and 7 attack categories. Tasks are primarily informed by published protocol-attack mechanisms.
Agents interact with running systems through scoped interfaces to achieve security-effect objectives without target source code. Three information levels assess guided reproduction, exploitation with mechanism and
topology context, and effect-directed black-box exploration. A hybrid verifier combines rule-based checks with LLM review of independently collected traffic and logs. We evaluate five LLMs. Claude Opus 4.8 achieves the highest observed success rates: 48.8% under L1 guidance and 12.5% under effect-directed black-box L3. In a separate DNSBomb optimization experiment, it achieves a peak downstream-response to average query-input byte-rate ratio exceeding 171,000 times. The results show that
agents can achieve verifiable exploitation effects in some cross-component environments, but success with limited black-box information remains low and varies across domains and attack categories. All experiments use controlled, explicitly authorized targets; they do not test production systems or disclose previously unknown vulnerabilities.
Read the full abstract ▾
All authors Fasheng Miao, Zhaoyu Hu, Xiang Li, Changqing An, Jiutian Zeng, Longtao Huang, Haixin Duan, Jilong Wang
Artificial Intelligence · Cross-listed from quant-ph
A Tutorial on Quantum Amplitude Embedding with Data Re-Uploading
Hyojun Ahn and Joongheon Kim
Amplitude
embedding lets a variational
quantum model read many classical features with few
qubits, but the model output is then only a quadratic form in the normalized input. Data re-uploading, which encodes the input repeatedly between trainable blocks, can be used to enrich such a model. Because every additional upload of an amplitude
embedding repeats a costly state preparation, the gain that an upload buys has to be known before the gates are spent. This tutorial thus addresses the problem of explaining what data re-uploading adds to an amplitude-embedded model and how large the gain is. We express sequential re-uploading and parallel re-uploading with two copies in one notation and show that each upload raises the reachable
polynomial order by two. Under one trainable
circuit family and one budget of trainable parameters, simulation results show that re-uploading raises the test coefficient of determination from 0.689 to between 0.883 and 0.975 on targets of order four and from 0.337 to between 0.642 and 0.783 on targets of order six, on all ten seeds. On a quadratic target and on handwritten digits, one upload remains the best choice, and every additional upload lowers the gate error that the classifier tolerates. These results help designers match the number of uploads to the
polynomial order of the task at hand.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Agentic-MatriBO: Capability-Compiled Agentic Bayesian Optimization for Materials and Chemical Design
Yu Zeng, Yuchuan Chen, Ziyue Zhang et al. (+5 more)
Materials and chemical optimization must choose the next synthesis, characterization, or high-fidelity simulation from sequential observations under a limited budget. Large
language models can use task knowledge to propose adjustments, but composing surrogates, acquisition functions, and search strategies is constrained by compatibility, parameter dependencies, and evidence requirements. We present Agentic-MatriBO, a capability-compiled agentic
Bayesian optimization (BO) framework that represents BO components and selection conditions as composable capabilities. A
language policy proposes BO recipes and candidate plans; a compiler checks proposals against the registered capability catalog; and a deterministic execution layer validates and commits candidates. Oracle observations are stored in research
memory and inform later rounds. We compare Agentic-MatriBO with eight baselines on four offline finite candidate pools—COF, PCE10, Buchwald–Hartwig (BH), and Suzuki–Miyaura—using five seeds per condition; results are reported descriptively. Across the four tasks, Agentic-MatriBO has the best mean endpoint quality after orienting each objective so that higher is better. On COF, it reaches the tabulated optimum in all five seeds both in the end-to-end comparison and after shared initialization, where it also retains an advantage in cumulative quality. Its normalized area under the best-so-far quality curve during the BO stage (nAUC-BO) is substantially higher on COF, near parity with the best baseline on PCE10 and BH, and lower on Suzuki. COF ablations show that removing the capability compiler or evidence gate coincides with lower cumulative quality and fewer exact-optimum hits. The framework provides an inspectable interface for evidence-conditioned BO adaptation.
Read the full abstract ▾
All authors Yu Zeng, Yuchuan Chen, Ziyue Zhang, Zijian Wei, Yanyi Su, Guojiang Zhao, Hongshuai Wang, Zhifeng Gao
Artificial Intelligence · Cross-listed from cs.SE
Diff Diffusion: Predicting Edit Paths with Diffusion Models for Efficient Code Editing
Xiaolong Xu, Hao Jiang, Bo Jiang et al. (+5 more)
Diffusion large
language models (dLLMs) are appealing for code generation because they can update many
tokens in parallel, yet many are optimized for sequence synthesis rather than edit prediction. This setting is poorly matched to modern AI-assisted IDEs, where edit prediction must infer the next code change from recent edits, the current file, and cursor-local context. Such changes are usually sparse and preserve most source
tokens, so regenerating a full code segment wastes computation. We propose Diff
Diffusion, a
diffusion-inspired LLM framework for code editing that predicts edit paths from an original snippet to its revised form. Diff
Diffusion represents denoising as structured edit operations, and combines this formulation with multi-stage training and efficient parallel decoding. By focusing computation on changed regions, Diff
Diffusion supports accurate low-latency edits and multi-position parallel modification. Across public edit-completion
benchmarks, edit-prediction evaluations, and online validation, Diff
Diffusion matches or leads autoregressive baselines while delivering 2.78 times faster inference on public edit-completion
benchmarks and a 39.0% P50 model-latency reduction in production. This work provides a new modeling perspective for AI applications that require high accuracy and low latency.
Read the full abstract ▾
All authors Xiaolong Xu, Hao Jiang, Bo Jiang, Zhiwen Deng, Ningyuan Sun, Bingzhou Chen, Jin Chen, Chenxing Wei
Artificial Intelligence · Cross-listed from cs.CV
QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editing
Tiancheng Shen, Zilong Huang, Xiangtai Li et al. (+6 more)
Multimodal Diffusion Transformers (MM-DiTs) have recently emerged as a powerful framework for unified text-
vision synthesis, surpassing traditional U-Net architectures in generative tasks. One key innovation lies in its
Multimodal Self-
Attention (MM-SA) interaction, where image and text
tokens are concatenated and processed via self-
attention. However, this mechanism poses significant challenges for editing, rendering conventional U-Net-based
attention manipulation methods ineffective. To address this limitation, we propose QK-Edit, a training-free framework that exploits the unique
attention dynamics of MM-DiTs for precise text-guided image and
video editing. By introducing a novel query-key manipulation strategy, our method isolates and adjusts critical
attention components to achieve an optimal balance between
prompt fidelity and structural consistency. This enables seamless edits across various tasks, including object addition, object removal, object replacement, changing background, changing material, changing color, and style transformation. Notably, it can be easily implemented with feature replacement in inference. QK-Edit demonstrates superior editing performance on state-of-the-art models, such as FLUX and HunyuanVideo, effectively bridging the gap between generative power and editable flexibility in MM-DiTs, and paving the way for scalable
multimodal content creation.
Read the full abstract ▾
All authors Tiancheng Shen, Zilong Huang, Xiangtai Li, Zhijie Lin, Jiyang Liu, Yitong Wang, Jiashi Feng, Ming-Hsuan Yang, Jun Hao Liew
Artificial Intelligence
Benchmarking and Training Reward Models for Cross-Source Evidence Grounding in Search Agents
Haochen Zhao, Minsheng Hao, Zheng Li et al. (+8 more)
Existing approaches to training and evaluating search
agents focus heavily on final-answer quality, largely neglecting how generated answers map back to retrieved evidence. Recent methods track citations to enforce grounding, but citation support alone does not establish source independence or whether an answer appropriately handles conflicting evidence. We argue that credibility in open-web
retrieval depends on principled cross-source evidence grounding that accounts for source independence and systematically attributes and adjudicates conflicting evidence, while hierarchical judgments of these grounding decisions provide a unified signal for training and evaluation. To this end, we propose an evaluation framework, Cross-Source Evidence Grounding (CSEG), that employs a fine-grained, six-layer decision tree over the complete search trajectory. We introduce CSEG-Bench and trainable reward models to provide targeted supervision following this framework. Our approach helps eliminating the severe data skew of natural trajectories by providing targeted coverage over complex, deep-layer decision paths, such as cross-source conflict resolution and nuanced attribution, that standard rollouts fail to capture. Specifically, (1) we define the hierarchy at label, node, and path levels; (2) we construct 1,160 verified trajectories over nine canonical paths by rewriting answers on real retrieved document pools, with independent frontier judges reproducing the entire gold path and human experts deciding inclusion; (3) we train CSEG-RM, which reaches 83.9% exact accuracy on our
benchmark while lying on the performance-
token efficiency Pareto frontier. Experiments that use this reward alone to supervise a frozen-
retrieval answer policy raise the independently judged mean label from 2.83 to 4.19, and show that learning the hierarchy generalizes across layers at a fraction of frontier-judge cost.
Read the full abstract ▾
All authors Haochen Zhao, Minsheng Hao, Zheng Li, Yifan Li, Zikang Liu, Bowen Qin, Yuhao Cheng, Chuang Fan, Haosheng Zou, Jeff Chen, Sujian Li
Artificial Intelligence · Cross-listed from cs.LG
Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR
Savyasachi Gupta and Aniruddha Srinivas Joshi
Reinforcement learning with verifiable rewards (RLVR) has become a principal recipe for improving the
reasoning of large
language models (LLMs). It rests on the assumption that the checker scoring each response is itself correct. In practice, rule-based checkers reject correct answers in unexpected formats, unit tests accept incorrect programs, and LLM judges can be led to approve empty or
adversarial responses. This work surveys 58 studies that measure, analyze, or mitigate such verifier errors. We ask what errors verifiers make, how they affect training, which mitigations work against which errors, and how the findings extend to self-generated, rubric- and judge-based, and verifier-free rewards. The evidence suggests that the structure of an error matters more than its rate. Random errors mostly slow learning, though under a fixed budget or KL regularization they also lower final accuracy. False positives that the policy can learn to trigger can cause plateaus or collapse, sometimes even with a low initial error rate. Whether a verifier wrongly rejects or accepts depends on what it checks and who built it, not on its family. Its errors shift as the policy improves, and most mitigations have been tested against only one kind of error. We compile the findings into comparison tables, reconcile apparent conflicts in the literature, and propose a reporting protocol to make future results comparable.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
Same Attack, Different Judge: A First Controlled Adversarial Comparison of Jev, LLM Judges, and Purpose-Built Safety Classifiers
Xiang Li, Aichun Huang, Mingming Zhang et al. (+1 more)
TypeSafe AI's Jev is a new kind of safety judge. It generates no text and no label: it answers questions the developer defines in advance and returns a probability for each answer. The vendor admits that
adversarial input can change these answers but gives no numbers. In this paper, we present the first controlled
adversarial evaluation of Jev. We build a test harness that sends every system the same payload under the same policy, and run it on four public-data tasks, from rule-based ones (input moderation, injection detection, routing) to a judgment-based one (phishing detection), with fifty attack variants from thirteen families. The targets are Jev, five GPT and GLM judges, and two local classifiers (Llama Guard 3,
Prompt Guard 2). We find that Jev is the most robust system on input moderation: eight forgeries of its structured input bypass none of its decisions, and a sentence asking only why someone might make a harmful request bypasses 9% of its blocks, against 56% for GPT-6 Astra. On phishing detection. However, the same forgeries bypass 19-29% of the emails it flags, and even without attack Jev detects only 62% of phishing emails, against 77-81% for the GLM judges. Jev is also the fastest judge, but it flags the most benign requests (27%). The local classifiers fail differently: a training-example statement makes Llama Guard 3 miss 62% of the phishing it had flagged. These results suggest that Jev resists attacks on tasks with a clear rule to check but is more exposed on tasks that call for an overall judgment.
Read the full abstract ▾
All authors Xiang Li, Aichun Huang, Mingming Zhang, Zheli Liu
Artificial Intelligence
Empowering Edge Agents: A Systematic Analysis of Edge-Cloud Collaboration in LLM Agent Execution
Pengcheng Huang, Zhenghao Liu, Shixin Ren et al. (+5 more)
Edge-cloud collaboration combines the lower inference overhead and greater data locality of edge-side execution with the stronger capabilities of cloud-hosted models. However, as LLM
agents increasingly operate in long-horizon scenarios, cloud participation becomes interleaved throughout execution rather than a one-shot decision for each query. Such interleaved participation introduces trajectory-level effects that remain insufficiently understood. To address this gap, we introduce EC2-Bench, a
benchmark for evaluating edge-cloud collaboration in long-horizon
agent execution. It provides executable digital-workspace
agent tasks and a unified evaluation framework covering task performance, resource usage, and
privacy exposure. Using this
benchmark, we empirically study diverse forms of edge-cloud collaboration. Our study reveals three findings. First, collaboration achieves a favorable balance among task performance, resource usage, and
privacy exposure compared with edge-only and cloud-only execution. Second, as edge models become stronger, collaborative execution approaches cloud-only performance, and performance differences across strategies shrink, while
privacy exposure remains strongly strategy-dependent. Third, stronger edge capability shifts preferred collaboration from externally controlled cloud participation and direct cloud execution toward more edge-controlled collaboration. Further analyses identify cloud-invocation decision-making and instruction following as important edge-side abilities for effective collaboration. These findings provide a systematic understanding of edge-cloud collaboration in long-horizon
agent execution and insights for designing future edge
agents that effectively collaborate with cloud models. Code and resources are available at https://github.com/OpenBMB/EC2-Bench.
Read the full abstract ▾
All authors Pengcheng Huang, Zhenghao Liu, Shixin Ren, Shi Yu, Sijia Yao, Yukun Yan, Chaojun Xiao, Maosong Sun
Artificial Intelligence · Cross-listed from cond-mat.mtrl-sci
Learning from fracture: Teaching language models to design through physical experience
Bo Ni
How can
language models learn to optimize physical designs through experience? We introduce FracLearn, an experience-abstraction-generalization framework evaluated through a deliberately unfamiliar, Tetris-inspired fracture game. The model first proposes architectures and examines simulated fracture responses to accumulate experience. Abstraction organizes these observations into conditional principles linking reinforcement placement, crack
evolution and load transfer, with supporting evidence and failure conditions. Generalization challenges the model to combine these principles in new architectures and surpass the earlier design record. Campaigns demonstrate progressive refinement and recovery through hypothesis revision, yielding a best composite score 11.0% above the experience-stage record. Fracture analyses connect improved performance to crack deflection and temporary trapping as toughening mechanisms, alongside greater residual load capacity. Learning occurs through external knowledge and decisions, without model-weight updates. This approach offers a route to scientific
agents that turn computational experiments into reusable physical knowledge for materials discovery and engineering design.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation
Xu Liu, Hanshu Yan, Taha Aksu et al. (+6 more)
Contextual forecasting seeks to improve time series forecasting by incorporating forward-looking information, such as planned events, external conditions, and expert expectations. However, reliable evaluation remains difficult: existing
benchmarks often have limited domain coverage, lack standardized history-future splits, or provide text that is not guaranteed to inform the future. We introduce GIFT-Eval-CTX, a
benchmark that isolates whether forecasting methods can translate verified future information into accurate numerical trajectories. Built upon GIFT-Eval, GIFT-Eval-CTX contains 10,000 instances across 20
datasets. Each instance provides structured effects derived from the realized future: Macro effects describe horizon-wide behavior, while Micro effects specify localized changes. We construct these annotations with an agentic pipeline combining LLM-based discovery and deterministic code validation. We
benchmark TSFMs, frontier LLMs, and tool-using agentic systems. The results show that structured effects reduce aggregate relative MASE by 40.7% for GPT-5.6 Sol and 29.6% for Claude Opus 4.8 relative to history-only prompting. The evaluated MoiraiAgent system achieves the best overall result. We further analyze the contributions of structured effects and agentic systems to performance, and identify opportunities for future research.
Read the full abstract ▾
All authors Xu Liu, Hanshu Yan, Taha Aksu, Quang Pham, Zhongzheng Qiao, Jun Hao Liew, Silvio Savarese, Doyen Sahoo, Junnan Li
Artificial Intelligence · Cross-listed from cs.LG
Posterior-Averaged Bellman Flow Matching for Distributional Critics
Zeqiao Li, Yijing Wang, Haoyu Wang et al. (+2 more)
Distributional critics model the full return distribution, providing richer signals for exploration and risk-sensitive control. Recent work has shown that flow-based critics can model these distributions flexibly and provide effective supervision across flow time for temporal-difference learning. However, unlike standard flow matching with fixed data endpoints, a flow critic is trained on Bellman targets recursively generated by a learned target critic under the current policy. Existing objectives therefore either use noisy single-endpoint supervision or rely on the learned flow itself for additional guidance. Our key observation is that an intermediate bridge state may be compatible with several Bellman endpoints, so its velocity should average their directions under the corresponding posterior. We propose Posterior-Averaged Bellman Flow Matching (PA-BFM), which samples an endpoint pool, infers the closed-form posterior over the latent anchor, and regresses to the posterior-mean velocity. Uniform anchor selection makes this target an exact finite-pool Rao–Blackwellization of the single-endpoint target: for every \(K\), it preserves the population target, cannot increase conditional variance, and its conditional variance scales as \(O(K^{-1})\). The endpoint pool supports conservative updates, while the twin return flows provide critic-disagreement and unsupported-mass diagnostics. Controlled studies show that PA-BFM reduces fixed-query target mean-squared error by \(79.7\%\) and target-induced gradient variance by up to \(98.0\%\), while improving return-distribution fidelity. Across 50 OGBench tasks and four offline-to-online scenarios, PA-BFM achieves higher aggregate performance than Value Flows.
Read the full abstract ▾
All authors Zeqiao Li, Yijing Wang, Haoyu Wang, Zheng Li, Zhiqiang Zuo