Artificial Intelligence · Cross-listed from cs.CR
CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering
Mark Russinovich
Indirect
prompt injection makes an LLM
agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that suppresses this behavior inside the model. Per model, a five-step recipe fits a residual-stream direction from paired episodes differing only in whether an embedded instruction is followed, and retains it only if it passes pre-specified causal and capability gates. At deployment, the direction is subtracted from every tool-result
token during prefill. The edit is always on–there is no detection decision to evade–and requires no fine-tuning, auxiliary model, or added
tokens, only white-box serving and tool-result span boundaries. Across five open-weights models (8B-106B, five vendor lineages), held-out attack success falls from 0.21-1.00 undefended to 0.00-0.17 defended, and AgentDojo compromise rate from 0.10-0.49 to 0.006-0.079, at 93-100% typography-normalized benign utility, with larger task-dependent costs when
reasoning over steered content. A
benchmark-level adaptive attacker reaching 0.67-0.73 undefended is held to roughly a quarter of that on the two most deeply evaluated models. Among the defenses we measured on capable models, those achieving lower compromise rates either lost 22-89% of benign utility or fine-tuned the served weights. White-box gradient attacks through the deployed vector compromise at most 2 of 52 episodes, and none of 2,052 replayed human red-team attacks succeeds. CounterSteer largely neutralizes instructional takeover: a black-box framing search cracks 3 of 18 development samples. Parameter manipulation–attacker-chosen arguments in otherwise legitimate calls–is only partially resisted (13 of 18); the decision becomes linearly readable at argument emission but not at the examined pre-generation sites, and is not removed by the tested prefill- or decode-time steering, motivating argument-provenance controls.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
From Checkpoint Variation to Selection Gains in Supervised Fine-Tuning
Yupeng Chang, Wenxuan Zhang and Yuan Wu
Checkpoint selection is a routine decision in supervised fine-tuning (SFT): training produces multiple checkpoints, but only one is retained. Yet fixed-budget comparisons do not by themselves distinguish three empirical claims: whether more validation data improve checkpoint selection, whether a selection rule outperforms validation-loss selection, and whether it improves over simply retaining the final checkpoint. We therefore treat checkpoint selection as a finite-information decision problem. Holding completed training trajectories, candidate checkpoints, and independent test items fixed, we vary the validation budget and separately measure improvement from additional validation data, gain over negative log-likelihood (NLL) selection, and gain over the final checkpoint. Across 60 mathematical SFT trajectories and 19 configurations, increasing the validation budget from 32 to 305-313 examples raises independent-test accuracy by 0.32 percentage points (pp) for generated-accuracy selection and 0.29 pp for checkpoint agreement, with 95% configuration-bootstrap CIs of [0.10, 0.56] and [0.11, 0.50], respectively. At the full validation budget, the two generation-based rules outperform matched NLL selection by 0.71 and 0.85 pp, respectively, while their gains over the final checkpoint remain unresolved. A cross-domain replication on 12 newly trained Commonsense trajectories shows the same qualitative separation: increasing the validation budget from 32 to 1,024 questions improves generated-accuracy and checkpoint-agreement selection by 0.87 and 0.27 pp, while gains over the final checkpoint again remain unresolved. Together, these results show that benefiting from more validation data, outperforming NLL selection, and outperforming the final checkpoint are distinct empirical claims that require separate evidence.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Models
Ruochen Zhang, Yao Huang, Yitong Sun et al. (+5 more)
While
Multimodal Large
Language Models (MLLMs) are increasingly deployed in safety-critical domains, their reliability is threatened by
multimodal implicit risks. Unlike explicit threats, these hazards emerge when individually benign text and neutral visual entities logically converge to induce unsafe outputs. Current detection methods fail to address this because they overlook the underlying risk activation mechanisms that govern cross-modal risk activation, leading to single-modality shortcut learning and hallucinated rationalizations. To bridge this gap, we first construct TriggerBench, the first
dataset explicitly modeling risk compositionality (5,600 instances). By formally isolating Key Elements and Trigger Elements to build counterfactual contrastive pairs, TriggerBench eliminates risk residues and forces models to perform genuine logical deduction rather than superficial pattern matching, which provides a rigorous foundation for both large-scale training and fine-grained evaluation. Building on this, we propose a Step-Supervised Structured
Reasoning training framework and employ it to train ThinkingGuard, a specialized guard model. Inspired by Situation Awareness theory, we decouple implicit risk identification into progressive cognitive stages, and utilize a step-reward Monte Carlo Tree Search
algorithm to explore optimal
reasoning trajectories, which are then distilled into the model through Dual-Constraint Preference
Alignment. Extensive experiments across both standard and implicit safety
benchmarks demonstrate that ThinkingGuard achieves strong performance. Project resources are available at https://github.com/FroggyChen/ThinkingGuard.
Read the full abstract ▾
All authors Ruochen Zhang, Yao Huang, Yitong Sun, Jiahe Xie, Jin Yan, Jifan Ma, Yuanfang Guo, Xingxing Wei
Artificial Intelligence · Cross-listed from cs.CV
FM-ReID: Selective Competitive Token Routing for Object Re-Identification
Zhiqi Li, Xiaowei Zhou, Zeyuan Sun et al. (+2 more)
Object re-identification (ReID) faces a recurring challenge: different identities can share highly similar global appearances, while the cues that distinguish them are localized, heterogeneous, and visible only under particular viewpoints. This challenge arises in animal ReID through markings, contours, and scars, in person ReID through subtle clothing and accessory cues, and in vehicle ReID through localized appearance details. Although visual foundation models encode such information in dense
tokens, a single holistic descriptor can obscure discriminative local signals. We propose FM-ReID, an end-to-end framework that formulates local representation learning as selective competitive
token routing. Its Competitive Fine-grained Mining module uses multiple mining queries and a residual query to compete for dense DINOv3
tokens. Above-prior selection retains
tokens preferentially allocated to each mining query, while the residual slot receives
tokens excluded from the
retrieval descriptors. The resulting multi-query descriptors are jointly trained with a holistic representation for
retrieval, without fixed spatial partitions or equal-area constraints. FM-ReID achieves strong results on animal, person, and vehicle ReID
benchmarks, supporting competitive
token routing as an effective way to augment holistic foundation-model representations.
Read the full abstract ▾
All authors Zhiqi Li, Xiaowei Zhou, Zeyuan Sun, Feng Gao, Junyu Dong
Artificial Intelligence · Cross-listed from cs.LG
HiTS-CL: A Continual Learning Framework for Long-Horizon Temporal Knowledge Graph Extrapolation
Yansong Liu, Rui Liu, Yuan Zuo et al. (+6 more)
Extrapolative temporal knowledge graph
reasoning (TKGR) predicts future facts from historical snapshots. Most existing methods train once on an early prefix of the timeline and then use a frozen model for all future timestamps. We argue that this fixed-prefix protocol is misaligned with extrapolation. It learns from a static prefix, whereas the target stream is non-stationary: new entities and facts emerge, temporal dependencies shift across regimes, and recurring historical signals must be refreshed online. As a result, models trained only on early snapshots become outdated and degrade over long horizons. We address this mismatch by formulating extrapolative TKGR as continual learning over streaming snapshots. Under this view, effective extrapolation must jointly handle current dynamics, stable knowledge, and recurring historical evidence. Based on these requirements, we propose History-enhanced Two-Step Continual Learning (HiTS-CL), a backbone-agnostic continual learning framework for extrapolative TKGR. HiTS-CL tracks current dynamics via continual fine-tuning, preserves stable knowledge via multi-teacher adaptive
distillation, and retains recurring historical evidence via a selective
memory of recent and frequent facts. We integrate HiTS-CL into five representative TKGR backbones and evaluate it on four
benchmark datasets. HiTS-CL consistently improves extrapolation accuracy, reduces long-horizon degradation, and outperforms strong continual-learning baselines, including a recent method for temporal knowledge graphs. Source code and data are available at https://github.com/liuyansong98/HiTS-CL.
Read the full abstract ▾
All authors Yansong Liu, Rui Liu, Yuan Zuo, Hongwei Zhao, Da Fu, Fuwei Zhang, Fuzhen Zhuang, Yong Chen, Zhe Li
Artificial Intelligence · Cross-listed from cs.CV
How Medical VLMs Underutilize Their Vision Encoders: A Dermatology Perspective
Janet Wang, Yunbei Zhang, Xiao Wang et al. (+1 more)
Medical
Vision-
Language Models (VLMs) show significant promise for clinical image understanding, offering accurate diagnosis with interpretable
reasoning. However, a critical performance gap exists between their strong
vision encoders and the full
multimodal model: in dermatology, the MedSigLIP encoder outperforms MedGemma by an average of 10.26 percentage points even when both use zero target-task labels; few-shot linear probing provides further evidence of strong visual representations. This gap motivates an investigation of how visual information is used in end-to-end diagnosis and why plausible-sounding predictions can lack grounding in image evidence. Using dermatology as our primary testbed, we systematically investigate three hypotheses for this phenomenon. We further provide a mechanistic analysis of the model's internal
attention patterns, showing that a simple describe-then-decide prompting strategy increases
vision attention by 30-40% during generation. Task-specific fine-tuning improves dermatology classification but reduces cross-domain medical question-answering performance in our evaluation. To address these challenges, we combine label-free prompting with low-label encoder-assisted reranking while keeping the VLM frozen. We validate the interventions across five VLM backbones in dermatology and provide supporting representation and
attention analyses across additional medical modalities.
Read the full abstract ▾
All authors Janet Wang, Yunbei Zhang, Xiao Wang, Jihun Hamm
Artificial Intelligence
MAADBench: The Refreshable Paradigm for Anomaly Detection in Multi-Agent Systems
Lei Ma, Dennis Hofmann, Haowen Xu et al. (+4 more)
Recent studies report that LLM-based multi-
agent systems (MAS) fail at rates of 41%-87%, yet to our knowledge, no
benchmark to date supports systematic anomaly detection (AD) for them. Building MAS AD
benchmarks is hard because they must remain fresh as LLM systems evolve: tasks may leak into training data and thus be memorized by LLMs, traces and anomaly patterns expire as backbones evolve, and labels must be provided reliably for each refresh. To address these challenges, we present MAADBench (MA: multi-
agent; AD: anomaly detection), the first refreshable MAS AD
benchmark designed for diverse, evolving LLM backbones underlying the
agents. MAADBench combines (1) sampled-and-coupled generative tasks over an approximately 10^37-task space to mitigate task leakage, (2) refreshable trace generation under configurable LLM backbones, and (3) automated provision of cost-free, deterministic step-level labels for fine-grained AD evaluation. Beyond offering the paradigm itself, we run MAADBench with five state-of-the-art LLM backbones and release the MAADBench-Full
dataset with 5,200 step-labeled traces. Benchmarking 25 AD methods on the MAADBench
dataset reveals substantial limitations in current approaches: they rely heavily on supervision, struggle with subtle MAS-specific anomalies, and lack robustness across LLM backbones. These gaps point to a rich research agenda for MAS-specific anomaly detection, with MAADBench providing a systematic and refreshable testbed for method development and evaluation. We open-source MAADBench-Full at https://huggingface.co/
datasets/hww123/MAADBench-full.
Read the full abstract ▾
All authors Lei Ma, Dennis Hofmann, Haowen Xu, Joshua DeOliveira, Peter VanNostrand, Lei Cao, Elke Rundensteiner
Artificial Intelligence · Cross-listed from cs.LG
SERA: Scale-Equalized Rollout Allocation for Maximum Likelihood Reinforcement Learning
Zihao Chen, Fanxiang Xiong, Hongran Ren et al. (+6 more)
Maximum Likelihood Reinforcement Learning (MaxRL) targets
prompt-wise log-success and has shown strong performance on
reasoning tasks. Under finite rollout budgets, however, the
estimator used by MaxRL attenuates each
prompt's likelihood gradient by a factor that depends on its success probability and rollout count. Under uniform rollout allocation, the common rollout count fails to compensate for success-dependent attenuation, leaving low-success
prompts more strongly attenuated and distorting their relative contributions to the expected aggregate gradient. We introduce SERA (Scale-Equalized Rollout Allocation), which redistributes a fixed rollout budget to approximately equalize these finite-rollout scaling factors. Building on our theoretical analysis of how finite rollouts distort
prompt-wise likelihood gradients, we formulate the allocation as a fixed-budget max–min problem, derive a waterline solution to its continuous relaxation, and introduce a multiplicity correction to remove the additional
prompt weighting induced by heterogeneous rollout counts. Experiments show stronger
alignment with exact likelihood gradients in a controlled ImageNet setting and improved multi-sample solution coverage over MaxRL on maze
navigation and mathematical
reasoning under matched training rollout budgets.
Read the full abstract ▾
All authors Zihao Chen, Fanxiang Xiong, Hongran Ren, Xuefeng Bai, Zhongxiang Dai, Kehai Chen, Zhiguo Zhang, Zhiyong Wang, Yu Cheng
Artificial Intelligence · Cross-listed from cs.LG
Interactive-Policy Distillation with Bidirectional Propose-and-Verify
Shutong Wu, Xiwen Chen, Brendan Rappazzo et al. (+4 more)
On-policy
distillation (OPD) trains a student model on its self-generated trajectories with dense
token-level teacher feedback. However, naive OPD may suffer from teacher unanchoring, where the student's
reasoning trajectory drifts far from the teacher, causing the teacher to be queried on states it would hardly visit and thus provide unreliable supervision. We propose Interactive-Policy
Distillation (IPD), which applies adaptive teacher intervention to the student rollout. Under a bidirectional propose-and-verify state machine, the student and teacher alternately exchange their roles as proposer and verifier, and collaboratively generate mixed-source trajectories. Then different supervisions are applied according to the source of each
token. This bidirectional propose-and-verify mechanism and the source-split loss make IPD not only a more performant
distillation method, but also a unified bridge between on-policy and off-policy paradigms. To make the interleaved dual-model rollouts more efficient, we also design a dedicated fused inference engine that co-hosts both models in one serving instance with separate KV caches and instantiates the state machine model to distribute, collect, and process requests. On math
reasoning tasks and across multiple teacher-student model pairs, student models trained with IPD not only outperform those trained with OPD, but also demonstrate higher data efficiency. Specifically, when distilling Qwen3-30B-A3B into Qwen3-1.7B-Base, IPD brings a +3.28 mean@8 and a +3.28 best@8
benchmark-averaged accuracy improvement compared with OPD. Besides, IPD only consumes about 1/4 of the training examples and steps to outperform OPD trained on the whole training
dataset for one epoch. We also investigate the impact of different loss variants and takeover / handback configurations, and demonstrate the robustness of IPD on different training data.
Read the full abstract ▾
All authors Shutong Wu, Xiwen Chen, Brendan Rappazzo, Daiheng Zhang, Anderson Schneider, Yuriy Nevmyvaka, Jiawei Zhang
Artificial Intelligence · Cross-listed from cs.CL
DraftTrace: A Multi-View Analytics Environment for AI-Integrated Writing
Divyansh Chandarana, Sandipan De and Vivek Gupta
Generative AI has changed how students produce writing assignments. The final artifact is no longer sufficient to understand the process through which it was produced. We introduce DraftTrace, a writing environment that jointly captures three complementary views of writing: the final product, the writing process and interactions with an integrated AI-assistant. DraftTrace reconstructs how a document develops over time and organizes these signals into submission, longitudinal, and class-level analytics for instructors. We deployed DraftTrace in a graduate NLP course with 81 students and compared their sessions with LLM-generated responses entered by automated tools and with copy-typed responses. While product measures distinguish differences in text formulation, process measures distinguish differences in how text is entered. Considering both views together helps characterize cases such as copy-typing. Interaction traces show that students use the assistant differently across stages of writing: to clarify the question at an early stage and to verify answers at a later stage. A preliminary instructor survey highlights the importance of multi-view writing analytics and their interpretability.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Adapting Context Compression for Long-Horizon Agents with Counterfactual Continuations
Guanghui Min, Liang Wu, Mingjia Shi et al. (+4 more)
Long-horizon
agents require context compression to manage growing interaction histories. Compression quality, however, is ultimately determined by downstream execution. Existing
prompt-adaptation methods infer compression errors by comparing full-context and compressed trajectories. Such comparisons cannot isolate individual compressions and are confounded by
agent stochasticity. We first find that compression degrades reliability before solvability. Using matched counterfactual continuations that compare execution from the same
agent state with versus without compression, we further show that severe degradation concentrates at isolated compression events. Motivated by this finding, we propose PAIR (
Prompt Adaptation using Interventional Rollouts) for adapting structured compression
prompts. PAIR identifies individual compressions that degrade subsequent execution, diagnoses their effects, and revises the relevant sections of a fixed compression template. PAIR achieves the strongest cross-run reliability among compressed methods in every main
benchmark-scope combination, consistently exceeding the competing
prompt-adaptation baseline. Without modifying the downstream
agent, PAIR brings compressed execution close to the no-compression baseline and sometimes numerically exceeds it.
Read the full abstract ▾
All authors Guanghui Min, Liang Wu, Mingjia Shi, Yinhan He, Mayank Darbari, Liangjie Hong, Chen Chen
Artificial Intelligence · Cross-listed from eess.IV
Reliability Testing of Medical Model Performance under Distributed Deployment
Yifei Wang, Xiaohan Zhang, Youtao Ding et al. (+4 more)
Distributed inference has become an indispensable part of deploying medical models under practical latency,
memory, and throughput constraints. Although modern frameworks improve serving efficiency through tensor parallelism, mixed precision, kernel fusion, and multi-device communication, they are generally assumed to preserve the behavior observed during centralized HuggingFace evaluation. This assumption creates an evaluation-deployment mismatch: a model may pass offline evaluation but produce a different output after the execution stack changes. To address this mismatch, we propose a testing framework and an improved, distributed-execution-sensitive medical-model
benchmark that evaluates the same checkpoint and input under a centralized HuggingFace reference and matched distributed deployments. Extensive experiments across
language,
vision, and
multimodal medical models show that execution changes can produce measurable output disagreements. Across supported visual settings, the test success rate ranges from 0.21 to 0.43 for single-modality models and from 0.32 to 0.98 for
multimodal models. The
benchmark is aimed at extending medical-model evaluation from capability and security to evaluation-deployment consistency.
Read the full abstract ▾
All authors Yifei Wang, Xiaohan Zhang, Youtao Ding, Tianlin Li, Xiaoyu Zhang, Yida Yang, Li Pan
Artificial Intelligence · Cross-listed from cs.RO
LIBERO-MAX: Do Robot Policies Adapt When the World Changes?
Yunbei Zhang, Zijian Jin, Yuanzhe Liu et al. (+14 more)
Robots must often continue a task after a target moves, the viewpoint shifts, or an obstacle appears, even though their earlier observations and committed actions reflect the previous scene. Many simulation robustness
benchmarks fix external conditions at reset, leaving this temporal challenge underexamined. We introduce LIBERO-MAX, a
benchmark of 8,000 paired cases spanning eight types of changes to geometry, observations, appearance, clutter, and paths. Each pair compares task execution with and without a mid-task event, holding the task, initial state, policy seed, and pre-event action sequence fixed. This controlled comparison distinguishes event-associated regressions from failures already present without the change. Across fourteen current VLA, hybrid, and world-action policies, events reduce success by 11.0-25.7 percentage points. Event profiles reveal shared vulnerabilities to geometry and observation changes, while policy-family rankings interleave. Camera controls show that robustness reflects both competence under the changed conditions and the trajectory from which they are encountered; varying query cadence does not eliminate the gap. Together, the paired protocol and temporal diagnostics establish LIBERO-MAX as a reproducible testbed for diagnosing failures under mid-execution changes and measuring progress toward
robot policies that remain effective as the world changes.
Read the full abstract ▾
All authors Yunbei Zhang, Zijian Jin, Yuanzhe Liu, Janet Wang, Xilun Zhang, Yuyou Zhang, Zhenyu Zhang, Daoan Zhang, Shuaicheng Niu, Gen Li, Jianfei Yang, Jihun Hamm, Ismini Lourentzou, Weirui Ye, Bo Liu, Peter Stone, Marco Pavone
Artificial Intelligence · Cross-listed from cs.CL
Large-scale factor analysis shows machine intelligence is only partially interpretable
Faiz Ghifari Haznitrama, Afrizal Hasbi Azizy and Faeyza Rishad Ardi
A common assumption in
language model development is that cognitive abilities are organized around a general, domain-free intelligence factor, like fluid intelligence in humans. This assumption is rarely tested directly, and prior attempts have done so only at a much smaller scale. We take a latent variable approach to intelligence in
language models, similar to how psychometricians study psychological constructs. Performance in every specific problem set is influenced by a domain-specific and a domain-agnostic latent factor. Using factor analysis as a dimension-reduction technique, we analyzed 13,251 published evaluation scores covering 1,618
language models across 456 different text-only
benchmarks. Due to the super-sparse nature of the
dataset, we triangulate our analysis across different data densifiers and imputation methods. A robust pattern across different modes of bias is that 1. A general intelligence factor accounts for 70.8% of variance in model performance at our most generous estimate, and far less than that in most of our solutions, 2. Content-similar
benchmarks do not necessarily cluster together, and 3. The $g$ factor is not dominated by any common theme, and there is a lack of evidence that it is well-proxied by standard "intelligence"
benchmarks. Our findings go against current endeavors of defining, identifying, and targeting general intelligence as a tangible construct in
language model development. This leaves the strategy of targeting a single conceptual ability without support, since the first-order abilities it would have to reach are often partially idiosyncratic and not identifiable in practice.
Read the full abstract ▾
Artificial Intelligence
BRIDGE: Bilevel Retrieval-Credit-Aware Agentic Reinforcement Learning
Quan Xiao, Mingda Liu, Gaowen Liu et al. (+2 more)
Agentic reinforcement learning (ARL) with verifiable rewards improves the ability of large
language models (LLMs) to tackle knowledge-intensive tasks by learning to interleave search and
reasoning. However, most existing ARL methods optimize only LLM-generated
tokens and treat retrieved evidence as environment observations. This creates an information-credit gap: failures caused by missing or misleading evidence are attributed to the LLM policy rather than to the retriever, which motivates training the LLM and the retriever jointly. In this paper, we show that
retrieval and LLM policy learning are order-sensitive: adapting the retriever before optimizing the policy yields a larger reward gain than the reverse order. To preserve this hierarchy while allowing both components to co-adapt, we formulate
retrieval-augmented agentic RL as a bilevel optimization problem. To solve it efficiently, we introduce BRIDGE, a
memory-efficient first-order bilevel method motivated by a loss-landscape analysis of the RL and
retrieval objectives. Across seven open-domain QA
benchmarks, BRIDGE achieves the highest average accuracy with both 3B and 7B backbones, improving the multi-hop average over the strongest baseline by 9.6 and 3.4 EM points, respectively. It also achieves the best averaged answer accuracy and
reasoning quality across medical QA
benchmarks.
Read the full abstract ▾
All authors Quan Xiao, Mingda Liu, Gaowen Liu, Katsuki Fujisawa, Tianyi Chen
Artificial Intelligence
AVIO: Learning to Add and Remove Sounding Objects in Audiovisual Scenes
Weihan Xu, Kan Jen Cheng, Koichi Saito et al. (+8 more)
Adding or removing a sounding object requires coordinated changes to visual content and sound while preserving the surrounding scene. Yet paired supervision for localized non-
speech audiovisual editing remains limited, as visual and acoustic edits must target the same object and isolate its sound from overlapping sources. To address this gap, we introduce AVIOBench, a
dataset comprising 37.9 hours of paired audiovisual examples spanning 1{,}878 target-object names. AVIOBench links the visual presence and acoustic contribution of each target object through a shared identity and visual mask. Our automated pipeline uses visual grounding and cross-modal consistency to select target-sound removal candidates, then jointly refines the audiovisual pairs to improve perceptual quality and cross-modal consistency. Building on this
dataset, we propose AVIO, which adapts a pretrained text-to audiovisual generation model through source-conditioned feature modulation to jointly learn object addition and removal. A reference-frame curriculum gradually reduces reference conditioning during training, enabling one model to perform instruction-only editing with optional visual guidance. Quantitative and qualitative evaluations demonstrate effective audiovisual object removal and addition, with optional reference guidance providing appearance and placement control for addition.
Read the full abstract ▾
All authors Weihan Xu, Kan Jen Cheng, Koichi Saito, Jingyu Shi, Tingle Li, Yisi Liu, Liming Wang, Masato Ishii, Takashi Shibuya, Gopala Anumanchipalli, Paul Pu Liang
Artificial Intelligence · Cross-listed from cs.HC
Towards Breaking the Learning System Wall Using Multimodal Tutoring Transcriptions
Danielle R. Thomas, Marie Cynthia Abijuru Kamikazi, Ashish Gurung et al. (+5 more)
Past research using log data has faced the "learning system wall," whereby few methods exist for generalizing models of student learning across platforms. Increasingly, online learning is captured by richer forms of data, including dialog and
video, with new affordances. An example of this is remote tutoring programs, where human tutors support students who use learning systems while
video conferencing. Toward better platform-general modeling of learning, we introduce an AI-driven
multimodal transcription system that processes screen-recording
videos into unified screenplay-style transcripts containing audio dialogue and annotated learning log actions. We describe a planned method for temporally aligning AI-generated
multimodal transcripts with MATHia learning logs and for identifying and classifying student learning processes to align with MATHia logs. Lastly, we highlight challenges and potential solutions in capturing learning processes in one system, offering initial steps towards generalizing log data across diverse systems.
Read the full abstract ▾
All authors Danielle R. Thomas, Marie Cynthia Abijuru Kamikazi, Ashish Gurung, Ishan Miglani, Shivang Gupta, Zachary Levonian, Conrad Borchers, Kenneth R. Koedinger
Artificial Intelligence · Cross-listed from cs.SD
InterBias-SV: Compound Conditions in Speaker Verification
Kamel Kamel, Hridoy Sankar Dutta, Keshav Sood et al. (+1 more)
Speaker verification systems encounter combinations of noise, channel distortion, and changes in
speech. Evaluating each condition separately does not establish whether their effects add. InterBias-SV organises this question around a four-term comparison: joint error, two marginal errors, and a common reference. Its results artefact contains 4,068 scored records across 17 experiments, 12 encoder labels, and six
speech corpora, totalling 12 million trial evaluations. Three experiment families contain the same-corpus terms needed to compute additive contrasts. For labels assigned to speaker-trained encoders, their mean contrasts are +0.0026, +0.0088, and +0.0024 in equal error rate (EER), with larger variation across settings. These descriptive averages do not establish equivalence to additivity: trial matching, checkpoint identity, and parts of the condition metadata remain unverified. We also examine two interpretation problems. Near-chance EER can make additive predictions difficult to interpret, but chance performance is not a hard EER ceiling, and correlation with the prediction does not identify a saturation mechanism. Ratios of demographic gaps are unstable when their clean reference is near zero; absolute gaps provide a more direct summary. The
benchmark provides condition definitions, analysis scripts, and explicit requirements for interpretable compound-condition comparisons, while separating recomputable summaries from claims that require further experimental validation.
Read the full abstract ▾
All authors Kamel Kamel, Hridoy Sankar Dutta, Keshav Sood, Sunil Aryal
Artificial Intelligence · Cross-listed from cs.CV
Benchmarking Vision-Language Models on Synapse Detection and Proofreading in Connectomics
Yicong Li, Junjie Wang, Leander Lauenburg et al. (+5 more)
We benchmarked
vision-
language models (VLMs) on the decisions annotators take when inspecting electron microscopy images in connectomics: synapse detection (presence and polarity) and proofreading (split errors and merge errors). For synapse detection, we evaluated 19 open and 2 closed models across various architectures and sizes under zero-shot, four-shot in-context learning and LoRA settings, against specialist models, on
datasets constructed by us using public resources. For proofreading, we evaluated 3 open and 2 closed models on the ConnectomeBench2
dataset, with cross-species transfer from fly and mouse to human and zebrafish. Most models were at chance zero-shot; a few examples helped mainly the closed and largest open ones. LoRA on a few thousand labels brought open models level with specialist models. When evaluated on unseen species, the best adapted VLMs outperformed specialist models trained on the same data in identifying merge errors. The project will be publicly available upon acceptance.
Read the full abstract ▾
All authors Yicong Li, Junjie Wang, Leander Lauenburg, Ella Hugie, Alexandra Irger, Wanhua Li, Donglai Wei, Hanspeter Pfister
Artificial Intelligence · Cross-listed from cs.LG
LLMs Learn to Evade Latent Monitors from Prior Feedback Alone
Hugo Lyons Keenan, Christopher Leckie and Sarah Erfani
Latent space monitors aim to detect undesired behaviors in LLM
agents by inspecting an
agent's internal activations rather than its outputs. However, interactive monitoring creates a feedback channel where each verdict the monitor delivers leaks information to the model about how its internal states are being evaluated. We ask whether an
agent can infer the monitor's decision rule from this feedback and then selectively edit its activations to evade detection. Unlike prior evasion attacks, the model is never explicitly told what the monitor detects. Surprisingly, off-the-shelf models already produce activation edits aligned with the monitored direction, but at insufficient magnitude for evasion. Simply scaling up these edits by a factor of 8 reduces the monitor's TPR from 100% to 27%. A rank-1 LoRA amplifies this behavior into effective evasion within the forward pass, reducing TPR further to 4% on held-out concept monitors while leaving other concepts at their normal detection rates. Capabilities on standard
benchmarks are retained under this finetuning, and the evasion skill survives retraining the monitors on the new activations. Mechanistically, we find evidence that the model computes its activation edit from the prior in-context turns, and show that the edit becomes more aligned with the monitored direction as more examples are provided. These results demonstrate feedback-conditioned control over activations and suggest that latent monitoring should be treated as an interactive process in which
agents can observe and respond to oversight measures.
Read the full abstract ▾
Artificial Intelligence
Human-AI Collaboration: From Paradoxes to Patterns
Michael Weiss
Evidence shows that humans and AI systems perform better together, by collaborating, than alone. This paper examines two key design dimensions of human-AI collaboration (autonomy and initiative) and explores the collaboration patterns that they generate. Documenting these patterns starts with identifying the underlying problems and solutions, followed by examining the internal tensions within the problems. The paper uses a paradox perspective to analyze those tensions. It describes a process for surfacing the tensions and mapping the underlying paradoxes. It also illustrates how the pattern descriptions can be derived from mapping these paradoxes. Finally, the paper documents four human-AI collaboration patterns: Instruction, Delegation, Assistance, and Co-creation.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from quant-ph
Quantum Computing for Network Security Classification: Near-Term Classification and Long-Term Memory Efficiency
Yuqing Li, Poonam Bala Nehru, Yunpeng Zhang et al. (+4 more)
Quantum computing has already been explored in several network-security applications. However, how
quantum computing may contribute to network-security classification in both the near term and the longer term has not been systematically discussed. This paper studies this question through two complementary experiments. First, we evaluate near-term
quantum-kernel support vector machines (SVMs) on practical network-security classification tasks and compare them with classical SVM baselines on KDD Cup 1999, CICIDS2017, and BoT-IoT. Across these runs,
quantum kernels are competitive. They can match or improve classical baselines in some settings, while classical RBF kernels remain stronger in others. This suggests that near-term
quantum-kernel methods should be evaluated as practical,
dataset-dependent alternatives to classical kernels rather than as uniformly superior replacements. Second, we use
quantum oracle sketching (QOS) to study a longer-term
memory advantage for classification with streaming classical samples. In QOS, samples are processed online and used to incrementally construct an approximate
quantum oracle, which provides coherent query access for downstream
quantum algorithms without retaining the entire
dataset. Under the QOS-inspired machine-size estimate, comparable accuracy corresponds to a substantially smaller effective
memory-size proxy than explicit sparse/QRAM-style storage. Compared with a simple streaming proxy, the result is more nuanced because aggressive feature filtering can make the streaming dimension small. This suggests that the long-term value of
quantum computing for network-security classification may lie in
memory-efficient data access rather than immediate runtime speedup. Together, these experiments show how
quantum computing may contribute to network-security classification from near-term classification performance and longer-term
memory efficiency.
Read the full abstract ▾
All authors Yuqing Li, Poonam Bala Nehru, Yunpeng Zhang, Danindu Gammanpilage, Xin Jin, Zeguan Wu, Junyu Liu
Artificial Intelligence
Learning to Harvest Without Collapse in a Regenerative Commons: A Lagrangian Framework
Jose Tupayachi, Xueping Li and Soham Das
The tragedy of the commons poses a multi-
agent safety problem: reward-seeking
agents can deplete a shared resource, and cooperation among its users does not itself specify how much must be preserved. We make preservation an explicit requirement by formulating a regenerative commons as a constrained Markov game or a constrained multi-
agent MDP with a designer-specified depletion budget. We develop a nonstationary Lagrangian framework that constructs a policy sequence from solutions of unconstrained games or cooperative control problems. Extending earlier time-average constructions, we introduce average-epoch solution concepts for reset episodes with discounted rewards and terminal costs. We prove a reward-independent feasibility certificate, cooperative feasibility and approximate optimality against feasible policy mixtures, and an extension to unbiased sampled costs. For self-interested
agents, a constrained Nash certificate quantifies the price-dispersion term introduced by deviations that redistribute budget across epochs. Under the stated assumptions on solver accuracy and multiplier updates, these results give constrained policy-sequence guarantees using solutions of unconstrained problems. Experiments with constrained IPPO and MAPPO in a Gordon-Schaefer fishery examine how depletion budgets shape stock retention, harvest rewards, and price adaptation.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Guard Models Are Overconfident Where Base Models Are Uncertain
Jonghyun Hong, MinJae Jung and Minwoo Kim
Guard models are used as safety classifiers, with confidence scores driving downstream moderation decisions. We evaluate five guard models for
prompt classification and find that although several are nearly calibrated on clean inputs,
adversarial attacks degrade their calibration by an order of magnitude, turning false negatives into high-confidence errors indistinguishable from correct detections. Comparing each guard with its corresponding base LM, we find that uncertainty signals often remain available, with the base model typically expressing uncertainty on the same inputs where the guard fails. Layer-wise analyses localize this guard-base divergence to later layers, where guard models exhibit sharper safe/unsafe separation and lower-rank representations, while
adversarial harmful inputs lie closer to the clean-safe region. These findings highlight a mismatch between guard confidence and base model uncertainty under attack.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Similar Choices, Different Attention: Cross-Modal Associations in Humans and Vision-Language Models
Sumin Hong, Katsumi Ibaraki, Renee Shi et al. (+2 more)
Cross-modal associations are systematic pairings of features across modalities, such as the association of 'bouba' with round shapes and 'kiki' with sharp shapes. Prior work has compared humans and
vision-
language models (VLMs) on such associations, but often using different stimuli or tasks between humans and models. Here, we ask whether VLMs align with humans not only in choices, but also in where they look when making those choices. We study both VLMs and humans (N = 53), presenting them with the same stimuli, a pseudo-word and two images, and record participants' choices and eye movements, which we release. We find choice
alignment in a few larger VLMs, but their saliency matches human gaze less closely than a center-bias baseline, a fixed Gaussian at the center of each image. Fine-tuning small VLMs on human choices brings their choice
alignment to the level of a human majority-vote reference on unseen words and images, yet their
attention still matches human gaze less closely than this baseline. Training model
attention on human gaze raises
attention-gaze correlation without improving choice
alignment, and a single average gaze map per image position raises it by a similar amount. Matching human choices, or even human gaze patterns, is therefore not sufficient evidence of human-aligned cross-modal processing.
Read the full abstract ▾
All authors Sumin Hong, Katsumi Ibaraki, Renee Shi, David Chiang, Toby Jia-Jun Li
Artificial Intelligence
Going Beyond State-Reaching: Learning Abstractions for Intrinsically Motivated Option Discovery
Akhil Bagaria, Anita De Mello Koch and George Konidaris
Temporal abstraction via options can improve exploration in large environments. However, existing option discovery
algorithms find subgoals that target all aspects of the state simultaneously. This state-reaching approach produces options that only apply in narrow regions of the state-space, eventually causing an explosion in the number of options that overwhelms the
agent, and impedes progress on its primary task of reward maximization. We introduce an
algorithm that instead identifies a small, relevant subset of features for each subgoal, yielding options that generalize broadly and accelerate exploration. Our approach learns abstract, transferrable options and achieves rapid exploration in three sparse-reward, image-based domains, including the Atari game MontezumasRevenge.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
Staircase Policy: Streaming Inference for World-Action Models with Large Action Chunks
Guoheng Sun, Chen Chen, Jin Wang et al. (+2 more)
World-Action Models (WAMs) improve robotic manipulation by conditioning action generation on predicted future observations, but future prediction adds further inference overhead to already expensive iterative action generation. Action chunking can amortize this cost over multiple actions, yet performance degrades over long execution horizons because later actions remain conditioned on stale observations. We introduce STAIRCASE POLICY, a streaming inference and training framework that turns a flow-matching VLA into a JEPA-style WAM and partitions a large action chunk into sub-chunks at staggered denoising stages. Near-term actions are executed as soon as they become available, while later actions continue to be refined. At each sub-chunk boundary, the future latent is re-predicted from the latest observation and used to update all unexecuted actions, enabling long-horizon execution without repeated full policy inference. The resulting future-prediction error can further serve as a signal for adaptive chunking. S-WAM achieves 97.7% on LIBERO and 87.9% on LIBERO-Plus, and improves performance across multiple policy backbones and real-
robot tasks. It reaches 292.7 executed actions per second, $3.62\times$ the throughput of conventional execution at comparable accuracy, while reducing time-to-first-action from 123.6 to 73.3 ms. With additional inference optimizations, throughput further increases to 642.9 actions per second.
Read the full abstract ▾
All authors Guoheng Sun, Chen Chen, Jin Wang, Ang Li, Teresa Lv
Artificial Intelligence
Rethinking Reasoning Paths as Phase-Structured Trajectories
Zhenghao He, Guangzhi Xiong, Sanchit Sinha et al. (+3 more)
Large
language models often improve problem-solving performance by generating multi-step
reasoning paths, yet how to analyze the hidden states along these paths remains unclear. Existing approaches typically assign each intermediate state the final-answer correctness label and train probes across heterogeneous questions. We argue that this protocol obscures
reasoning dynamics in two ways: (1) correctness prediction can exploit question-level variation rather than path quality, and (2) states aligned by absolute step indices may correspond to different functional phases of
reasoning. In this work, we propose to view
reasoning paths as phase-structured trajectories within fixed questions. We instantiate this view as PAIR, short for Phase-Aligned Intra-question
Reasoning. PAIR samples multiple trajectories for each question, maps variable-length paths into shared relative phases based on normalized trajectory progress, and compares successful and unsuccessful trajectories only within the same question and phase. This yields phase-specific path-quality directions that better isolate path-quality signals from question-level variation. Empirically, we find that standard across-question correctness probes lose much of their predictive power under within-question evaluation, suggesting that these probes partly rely on question-level information. PAIR improves within-question trajectory ranking and Best-of-N trajectory selection across models and
benchmarks. Phase-wise steering further shows that the learned directions can change generation outcomes, providing causal evidence that they capture trajectory-relevant information.
Read the full abstract ▾
All authors Zhenghao He, Guangzhi Xiong, Sanchit Sinha, Bohan Liu, Wenqian Ye, Aidong Zhang
Artificial Intelligence · Cross-listed from cs.SD
Emergent Tonal Structure in Learned Chord Embeddings and Its Relation to Tonal Tension
Maral Ebrahimzadeh, Gilberto Bernardes and Sebastian Stober
Several tonal pitch spaces and computational models have been proposed to analyze tonal structure in Western tonal music, many of them grounded in principles from music theory and used to support tonal analysis with important implications for tonal tension. In parallel, data-driven methods such as skip-gram have been used to learn chord
embeddings from symbolic corpora, but their ability to recover tonal structure and its relation to tonal tension remains underexplored. In this work, we investigate how skip-gram chord
embeddings reflect tonal structure and whether they provide a useful basis for analyzing structural aspects of tonal tension. Using chord sequences with and without transposition-based augmentation, we evaluate the learned spaces from
geometric, functional, and tension-related perspectives. We show that augmented
embeddings exhibit strong transposition equivariance, recover a clear circle-of-fifths structure, and support interpretable shifts between key-related regions of the learned space. We then derive
embedding-based measures from chord-to-key distance and contextual chord-distance relations, and show that they capture meaningful aspects of tonal tension structure through correspondence with matched tonal measures and moderate
alignment with human tension profiles. Across analyses, transposition-based augmentation generally improves the stability, tonal coherence, and interpretability of the learned space.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Channel-Dependent State Space Model for Multivariate Time Series Forecasting
Yu-Cheng Wu, Fan-Keng Sun, Li-Chun Lu et al. (+1 more)
Multivariate time series
forecasting (MTSF) is critical across many real-world domains. Existing deep learning approaches fall into two paradigms with distinct limitations: channel-independent (CI) methods unconditionally ignore cross-variable dependencies and model only temporal dynamics, while channel-dependent (CD) methods consider both but typically rely on architectural compromises to mitigate overfitting and computational overhead. We therefore propose Chameleon, a specialized CD state space model (SSM) that enables data-dependent, fine-grained interactions across variables while scaling linearly with their number. By connecting selective SSMs with the Kalman filter, we leverage the missing measurement update in the former for cross-variable modeling while preserving the SSM backbone for robust temporal modeling. We further identify favorable inductive biases of GatedDeltaNet for time series, adapt it as our backbone, and improve generalization through additional techniques, including a previously unexplored
stochastic perturbation of reversible instance normalization. On strongly dependent ODE and PEMS
datasets, Chameleon achieves the best MSE and MAE across all settings, while its CI ablation and prior CD methods incur 61-178% higher MSE on average. Across 28 standard
benchmark settings, Chameleon also achieves better MSE and MAE than each baseline in at least 27 and 22 cases, respectively. Training-time and peak-
memory analyses on Traffic and ETT further demonstrate competitive efficiency and favorable
memory scalability across different variable counts.
Read the full abstract ▾
All authors Yu-Cheng Wu, Fan-Keng Sun, Li-Chun Lu, Duane S. Boning