Artificial Intelligence
Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors
Arya Labroo, Mengjie Qian and Kate Knill
Automatic speaking assessment systems are increasingly deployed in high-stakes settings to mark second
language (L2) learners' speaking tests, making it critical to show that their scores depend on speaking proficiency rather than irrelevant speaker attributes such as first
language (L1) or age.
Transformer-based foundation models have improved the accuracy of these L2 speaking graders, but their black-box representations make
fairness and interpretability analysis more difficult. Building on prior work that used Concept Activation Vectors (CAVs) to detect bias towards unwanted attributes (`concepts') in feature-based graders, we extend CAV-based analysis to two neural speaking assessment systems: a text-based BERT grader and a
speech-and-text
multimodal grader based on Whisper. CAVs represent human-interpretable concepts as directions in a model's activation space, allowing us to distinguish between whether a concept is encoded in a model's internal representations and whether it influences the predicted score, the latter quantified using a gradient-based sensitivity metric. Since CAVs rely on linear separability, which is less likely in complex neural
embedding spaces, we also investigate whether sparse autoencoders (SAEs) provide cleaner concept directions by learning CAVs in a sparse latent space and mapping them back to activation space. Our analysis shows that concept recoverability depends strongly on the representation and architecture being probed, rather than on the concept alone. Sensitivity to concepts is also architecture-dependent. SAEs make concepts more linearly recoverable, but attenuate the original activation-space sensitivity, especially in low-dimensional layers. These findings highlight the need to distinguish concept recoverability from concept influence when
auditing bias in speaking assessment systems.
Read the full abstract ▾
Artificial Intelligence
QuanTiMedAI: Quantum-Enhanced Time-Series Model guided by Agentic AI for Cardiac Arrest Mortality Prediction
Mutasim Fuad Sarker, Adiba Rahman Namira, Wafa Binte Alam et al. (+3 more)
Cardiac arrest remains one of the most lethal conditions encountered in intensive care units. Despite the growing availability of electronic health record data, existing mortality prediction studies in this population largely depend on static summaries derived from early admission. Such approaches ignore the temporal progression of physiological deterioration and recovery that unfolds throughout a patient's ICU stay. To address this limitation, we introduce QuanTiMedAI, a
quantum-agentic framework developed for cardiac arrest mortality prediction using agentic AI guided
quantum enhancement time series model. The proposed system combines an agentic large
language model (LLM) for clinically informed feature discovery with a compact
quantum recurrent network for temporality aware mortality prediction. Our findings demonstrate that agentic LLM-guided feature selection consistently outperforms conventional feature selection approaches, and the proposed
quantum architecture achieves competitive predictive performance through nonlinear feature enhancement while keeping the number of parameters very low. Through extensive experimentation on a MIMIC-IV cohort of cardiac arrest patients, QuanTiMedAI's
quantum-enhanced architecture attains an AUROC of 0.852 using only 605 parameters, an improvement of approximately 2.9% over a current state-of-the-art baseline for this task. A structured ablation study systematically validates the contribution of each architectural design choice. These results show that
quantum-enhanced sequential modeling can exceed classical recurrent networks while using substantially fewer parameters.
Read the full abstract ▾
All authors Mutasim Fuad Sarker, Adiba Rahman Namira, Wafa Binte Alam, Md Adnan Arefeen, Mahzabeen Emu, Sumaiya Tabassum Nimi
Artificial Intelligence · Cross-listed from cs.LG
FastKron: Efficient Quantization with Kronecker-Factored Hessians
Johann Birnick and Rayan Saab
We accelerate a family of
algorithms for neural network quantization which utilize a two-sided version of the GPTQ/LDLQ
algorithm. Standard GPTQ-style adaptive rounding uses one-sided correlation information derived from input activations. A natural two-sided extension can additionally capture correlations across output channels. It utilizes a general Kronecker-factored approximation of the weight
matrix curvature. This approach has been used in BoA and YAQA. But making a concrete algorithmic implementation of this two-sided GPTQ variant is nontrivial. BoA uses a large number of sequential steps, while YAQA improves over the sequential depth but still has quartic total cost. We introduce FastKron, an efficient algorithmic implementation that combines anti-diagonal parallelism with a recursive divide-and-conquer construction. For an $m\times n$ weight
matrix, FastKron uses $O(m+n)$ sequential steps while reducing the total work from $O(m^2n^2)$ to $O(mn(m+n))$. Thus, it matches the cubic scaling of GPTQ while exploiting richer
curvature information. Moreover, FastKron is modular with respect to both the base quantizer and the Hessian
estimator. We also provide practical
benchmarks, consider a range of Hessian approximations that FastKron can be used with, and provide an efficient technique to compute these Hessians.
Read the full abstract ▾
Artificial Intelligence
The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images
Zhiheng Wang, Bo Peng, Lai Wei et al. (+1 more)
The "thinking-with-images" paradigm equips
multimodal LLMs with active visual operations such as crop-and-zoom. However, models using these operations often achieve only marginal or negative gains over direct inference at substantially higher
token cost. They may also repeatedly crop irrelevant regions and fail on questions that direct inference answers correctly. We ask whether the returned visual evidence causally affects the answer. To answer this question, we formulate visual tool-use as a causal graph that separates observation-mediated paths from action-induced shortcuts. We then audit it through interventions at the three levels: policy (comparing tool-use with direct inference), trajectory (corrupting all observations during rollout), and step (counterfactually replacing one individual observation under a fixed prefix). Our step-level estimand, Visual Evidence Gain, isolates the contribution of each returned observation. Across six representative models and five fine-grained perception
benchmarks, we uncover policy miscalibration with two failure modes. In Calling Without Looking, returned observations have no causal effect on the answer. In Looking Without Planning, observations are informative but the call schedule is incoherent. A trajectory-level diagnostic decomposes the policy-level accuracy gain into per-group contributions and shows that the gain is concentrated in a Calibrated minority. We term this discrepancy the illusion of visual tool-use: despite aggregate accuracy gains, visual tool-use is not causally effective across a broad range of rollouts. The code is available at https://github.com/OpenCausaLab/CauAudit.
Read the full abstract ▾
All authors Zhiheng Wang, Bo Peng, Lai Wei, Chaochao Lu
Artificial Intelligence
Improving the Realism of Synthetic Clinical Benchmarks Under Utility Constraints
Omid Bazgir, Md Nasir, Jacob Hoffman et al. (+6 more)
Synthetic clinical
benchmarks for enterprise AI
agents can pass existing utility checks and still remain structurally unrealistic, especially in
privacy-sensitive healthcare settings where operational data are hard to access. We study how to improve such
benchmarks without breaking the downstream utility checks already used in practice. We formulate
benchmark revision as utility-constrained realism improvement:
dataset changes should increase realism while staying above an operational utility floor. We instantiate this idea on a care-gap
benchmark derived from Synthea-generated patients exercised through demonstration electronic health record workflows and then processed by the same downstream pipeline as operational data. Realism is measured through missingness structure, simplicity, structural plausibility, and population
alignment. The baseline
benchmark is extremely thin: sampled-pair missingness is 79.44%, only 12.75% of rows are actionable, 38.94% of patients have zero actionable measures, and top-three
token concentration reaches 100.0%. Two deterministic revisions improve these panels while remaining above the current utility floor, whereas a naive densification control preserves unrealistic templating. We further show that internal
benchmark realism and source fidelity to an aggregate operational reference are related but distinct objectives. These results suggest that synthetic
benchmark quality should be optimized explicitly, with utility treated as one constraint rather than as sufficient evidence of realism.
Read the full abstract ▾
All authors Omid Bazgir, Md Nasir, Jacob Hoffman, Yang Yang, Manu Agrawal, Anusua Trivedi, Vinay Rao Dandin, Chris Gibbons, Christine Swisher
Artificial Intelligence · Cross-listed from cs.CV
Toward Deployable Bangla Sign Language Recognition with Expert-Validated Data and a Lightweight Attention-Based Model
Saad Ahmed and Md Khalid Syfullah
Deaf and hard-of-hearing people in Bangladesh communicate mainly through Bangla Sign
Language (BdSL). Automatic BdSL recognition on personal devices could widen access to education and services. Existing systems use controlled-setting
datasets without expert verification and heavyweight pretrained backbones unsuited to on-device use. We introduce RSBdSL38, 10,874 expert-validated images spanning all 38 BdSL hand signs, representing the 51 letters of the Bangla alphabet, recorded from real signers at three special-needs schools across Bangladesh. We propose a lightweight
attention based convolutional network of 298,470 parameters, built from grouped bottleneck residual blocks, channel and spatial
attention, a multi-scale depthwise hand-feature block, dual pooling, and Swish activations. Trained from scratch, it attains 96.37% accuracy (95.72% +- 0.54% over five seeds), within 1.08 percentage points of the best of nine ImageNet-pretrained efficient architectures under an identical protocol, using 8.5 to 68x fewer parameters and 1.3 to 21.7x fewer MACs. Retrained, it reaches 92.95 to 98.33% on six public BdSL
benchmarks, 97.04% on a merged corpus, and 76.25% zero-shot on BdSL-38. Removing any architectural stage costs 7.61 to 89.30 points, against at most 3.17 for the training recipe. Grad-CAM with deletion-insertion and weight-randomization checks confirms that predictions follow the signing hand. A signer-independent split holding out 6 of 36 signers yields 85.18%. Quantized to 0.48 MB, it runs at 3.98 ms per image within a 15.5 MB footprint on a commodity smartphone. Together, RSBdSL38 and our from-scratch model turn
benchmark accuracy into deployable accessibility at a fraction of pretrained-backbone cost;
dataset, code, and models are released.
Read the full abstract ▾
Artificial Intelligence
DASH: Divergence-Adaptive Supervision Horizons for On-Policy Self-Distillation of Reasoning Models
ZhiYan Hou, Xinyu Tang, Hongyan An et al. (+9 more)
Reinforcement learning with verifiable rewards (RLVR) improves the
reasoning capabilities of large
language models using automatically verifiable outcome signals, but these signals are typically sparse and at the sequence-level. On-policy self-
distillation (OPSD) mitigates this sparsity by querying a privileged teacher at student-visited prefixes and providing dense
token-level distributional supervision. Although this dense supervision alleviates signal sparsity, we find that standard OPSD still underexploits the temporal structure of the rollout. It assigns every local divergence the same coefficient, regardless of its position or the divergence sequence in which it occurs. In on-policy autoregressive generation, the same divergence magnitude can follow different discrepancy histories, reflecting different evolutions of the mismatch between the teacher and student. Since the local scalar alone cannot distinguish these temporal contexts, standard OPSD cannot adapt its
token-level weights to the realized discrepancy sequence. To address this limitation, we propose Divergence-Adaptive Supervision Horizons (DASH). DASH maps the gap between each local
distillation signal and the sequence-level mean to an adaptive propagation gate and then uses these gates to control backward multi-step aggregation. By doing so, DASH adjusts
token-level supervision weights according to how local divergences evolve during generation. Experiments on three mathematical
reasoning benchmarks across three model scales show that DASH improves over our matched vanilla OPSD reruns on every
benchmark at all three scales. DASH reuses the teacher and student distributions that OPSD already computes, so the gains require no additional teacher or student forward pass. Code: https://github.com/DBtxy/DASH-OPSD
Read the full abstract ▾
All authors ZhiYan Hou, Xinyu Tang, Hongyan An, Jianjin Zhang, Weizhen Wang, Yunyun Han, Gengsheng Li, Xiangzhao Hao, Haiyun Guo, Wenbin Hu, Jinqiao Wang, Yafeng Deng
Artificial Intelligence · Cross-listed from cs.CV
PRISM: Distribution-Gated Flow Matching for Controllable Unpaired Image Translation
Elad Yoshai and Natan T. Shaked
Unpaired image-to-image translation must decide, per image, what to change and what to preserve without paired supervision. Many
diffusion-based unpaired translators control preservation through a single global noise or guidance value applied across the image, which cannot separate content to keep from appearance to change. We present PRISM, a GAN-free flow-matching framework that replaces this global control with a learned per-feature gate. The gate's spatial prior is derived from each source feature's standardized distance to the target feature distribution, so features far from the target are freed while target-consistent features are preserved. The same gate controls both the initialization, which mixes the real source latent with a task-matched corruption, and the transport timing during Ordinary Differential Equation (ODE) integration. The corruption is matched to the task, content-anchored (AdaIN) for structure-preserving translation and partially anchored for structure-changing translation, and the gate can be overridden locally at inference from text or a detector without retraining, preserving important structures of the original image while still generating realistic results. We evaluate PRISM on five natural and biomedical
benchmarks (AFHQ cat->dog, CelebA-HQ appearance translation, day->night relighting, virtual staining, and breast frozen->permanent histopathology). Among the evaluated methods under a shared same-split protocol, PRISM attains the best Inception FID and KID on four
benchmarks and a competitive result on the fifth, and on histopathology yields the
nuclei-count ratio closest to the ideal, supporting a favorable balance between target realism and structural preservation.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Depth-Guided Video Object Counting in Crowded Scenes
Yuanjing Xu, Xinyan Liu, Weidong Chen et al. (+5 more)
Our primary objective is to advance
video object counting in crowded scenes, aiming to robustly count all instances of a target category based on given text or visual
prompts. Existing methods rely on RGB information, limiting their discriminative ability in crowded and occluded conditions. To address this, we propose a Depth-Guided Detector (DG-Det) along with a general post-processing pipeline. By integrating depth cues with multi-scale RGB-D cross-
attention and explicit occlusion prediction, our method enhances spatial understanding and achieves robust detection in crowded and occluded scenes. Furthermore, we introduce a unified de-duplication framework to eliminate cross-frame redundant counting. To facilitate future research, we also release a new RGB-D
Video Object Counting
dataset featuring depth information and multiple object categories persequence. Extensive experiments demonstrate that our method achieves a 62.01% reduction in MAE compared to existing baselines, and also produces consistent improvements in RMSE. We provide the source code at https://github.com/streamer-AP/DG-Net and the
dataset at https://huggingface.co/
datasets/aerospace123/RGBD-VideoCount.
Read the full abstract ▾
All authors Yuanjing Xu, Xinyan Liu, Weidong Chen, Zixuan Zou, Linhao Zhang, Zhuangzhe Meng, Antoni B. Chan, Weigang Zhang
Artificial Intelligence · Cross-listed from cs.NI
From Passive Mirrors to Active Agents: Holonic Digital Twins for Physical AI over Networks
Christo Kurisummoottil Thomas, Omar Hashash and Walid Saad
Despite advances in artificial intelligence (AI) across multiple sectors, today's AI tools, including deep learning and generative AI, still fail when embedded into physical systems, such as
robots and vehicles operating under real-world physical laws. This stems from their inability to maintain reliable world models for long-horizon planning under uncertainty and generalize to unseen scenarios. In this context, wireless networks, through pervasive sensing and communication, can orchestrate physical intelligence. However, current architectures optimize throughput, latency, and reliability and cannot support real-time physical AI coordination, requiring
agents to maintain shared spatiotemporal context. To address these challenges, a network of holonic digital twins (HDT-Nets) framework is proposed to deliver real-time physical AI inference through holonic
agents that actively reason about their environment rather than passively mirror physical assets. Each HDT is realized as a hierarchical structure spanning the physical
agent and network edge,
reasoning autonomously at the local level while cooperating with neighboring HDTs to form collectively intelligent units. In HDT-Net, causal Markov blankets spanning sensing, communication, and control determine which
agents must coordinate and enable counterfactual
reasoning over multi-domain interventions. Active inference within these boundaries unifies perception, action, and learning by minimizing expected free energy while deciding which beliefs to transmit based on their cognitive value to the receiver. Category theory ensures that transmitted beliefs preserve semantic structure across heterogeneous
agents with incompatible representations. Finally, integrated information theory quantifies when collective intelligence exceeds independent operation and how network intelligence evolves through coordinated learning and information exchange.
Read the full abstract ▾
Artificial Intelligence
TS-RAG: Retrieval Augmented Generation for Time Series Forecasting
Yixiong Xiao, Congxi Xiao and Jingbo Zhou
While deep learning models, particularly
transformer-based architectures, have shown impressive performance in time series
forecasting, the application of
retrieval-augmented generation (RAG) in this domain remains limited. Since RAG has proven effective in enhancing the capabilities of large
language models by incorporating relevant external information, retrieving similar time series sequences as references might also improve accuracy in time series
forecasting tasks. However, most time series models are constrained by limited training data, smaller parameter scales, and a lack of the extensive generative capabilities found in large
language models. Simply concatenating reference sequences into the
prompt, as done in
language models, may not yield the expected results. To address these challenges, we propose a novel approach, TS-RAG, which leverages RAG to enhance
forecasting performance. The framework introduces specially designed reference
tokens to effectively fuse information from the input sequence with that from retrieved similar sequences, enabling a more robust capture of complex temporal dynamics. Experimental results demonstrate that TS-RAG achieves consistent state-of-the-art performance across several real-world
forecasting benchmarks.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Continual Learning in Transition
Zhiyan Hou, Dan Zhang, Tao Feng et al. (+11 more)
Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.g., training strategies, architectural designs, and weight adaptation. However, emerging paradigms are reshaping the scope of CL beyond this traditional model adaptation view. For instance, on-policy learning broadens the space of update mechanisms; test-time training extends CL from the training phase to inference; and external harness components such as
memory, skill libraries, and interaction protocols extend the evolutionary boundaries of model capabilities far beyond the static parameter space. Collectively, these developments indicate a transition from parameter-centric learning toward system-level adaptation. To characterize this transition, we examine the
evolution of continual learning through three dimensions: When, How, and Where learning occurs. The How dimension encompasses off-policy, on-policy, and beyond-gradient optimization mechanics. The When dimension captures
evolution across pre-training, post-training, and inference-time stages. The Where dimension delineates updates occurring within internal parameters versus external structural constraints. Anchored by this tri-axial framework, we systematically survey representative methods, trace the ongoing transition of continual learning, and discuss the key challenges, broader implications, and future directions arising from this paradigm shift.
Read the full abstract ▾
All authors Zhiyan Hou, Dan Zhang, Tao Feng, Liyuan Wang, Wei Li, Xiangzhao Hao, Hongyan An, Junfeng Fang, Haokai Ma, Zhaohui Xu, Xinyu Tang, Haiyun Guo, Jinqiao Wang, Tat-Seng Chua
Artificial Intelligence · Cross-listed from cs.HC
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
Ro Encarnación, Tina Behzad, Emma Lurie et al. (+1 more)
Large
language model (LLM)
benchmark evaluations are routinely used to support claims about model safety, reliability, and deployment readiness. Yet most evaluations rely on a single access modality (model APIs), perform a single run per
prompt, and report accuracy as the primary outcome metric, without accounting for conditions such as web search that may have effects on model behavior in deployment. We audit these assumptions for one of the most widely-used LLMs, comparing two modalities, ChatGPT's chat UI and OpenAI's API, with and without web search enabled. We use a stratified total sample of 401
prompts from two popular
benchmarks, BBQ and SafetyBench, collecting 4,812 total responses across three repeated runs per
prompt. Beyond standard performance measures, we evaluate model output dimensions including response consistency, response text similarity, citation grounding, and abstention behavior. For instance, chat UI responses were less accurate than API responses on both
benchmarks with search disabled. Enabling web search reduced accuracy by up to 8 percentage points, and even reversed the direction of modality performance trends for one
benchmark. Repeated runs of the same
prompt produced inconsistent responses in up to 21% of
prompts. The two modalities also grounded answers in different citations, and abstention behavior was also inconsistent across both modalities. These results illustrate that, even within a model family, reporting only simple accuracy metrics can obscure important forms of model behavioral variation relevant to AI safety assessments. We argue that AI safety evaluations should systematically account for modality, multi-run consistency, search conditions, and response-level behaviors to better reflect how deployed AI systems behave in practice.
Read the full abstract ▾
All authors Ro Encarnación, Tina Behzad, Emma Lurie, Danaé Metaxa
Artificial Intelligence
EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning
Zishan Xu, Zhiyuan Yao, Yuxin Chen et al. (+9 more)
Training large
language model
agents for long-horizon tool use typically relies on interactions with real or synthesized executable environments, whose construction and verification are costly, or on external simulators that are difficult to ground. We introduce EnvACE, an agentic reinforcement learning method that replaces external environment interaction during training with world rehearsal. The policy alternates between acting and rehearsal: it first generates a tool call, then plays the role of the environment to produce the response induced by that action, and conditions subsequent decisions on the rehearsed response. Both roles are jointly optimized end-to-end using task-success rewards. Through world rehearsal, the policy internalizes the relationship between actions and their environment responses in its parameters, yielding an
agent world model that directly supports decision making. Across BFCL-v4, tau^2-Bench, VitaBench, and FinMCP-Bench, EnvACE achieves strong and transferable performance, outperforming environment-scaling baselines in the overall evaluation. Controlled studies further show that world rehearsal consistently improves policy learning across model scales. At test time, the internalized world model enables private rehearsal before committed execution, yielding further gains under a moderate rehearsal budget without additional external interaction. Our findings establish world rehearsal as a new path toward scaling LLM
agent training beyond the constraints of external environments. Our code is publicly available at https://github.com/Within-yao/EnvACE.
Read the full abstract ▾
All authors Zishan Xu, Zhiyuan Yao, Yuxin Chen, Yifu Guo, Zhengxi Lu, Yuquan Lu, Jinyang Huang, Yan Xu, Yasheng Wang, Weinan Zhang, Xingshan Zeng, Weiwen Liu
Artificial Intelligence
Comparative Approaches to Agent Retrieval over Large Skill Libraries
Indivara Kolluru and Nathan Sportsman
Agents backed by large skill libraries must decide which skills to load and in what order. Loading the entire library into context is expensive and provides no structure for autonomous sequencing. We study two systems for this problem over a corpus of 690 skills: a hybrid ranker combining lexical and dense-
embedding retrieval for sparse, on-demand loading, and a typed knowledge graph encoding workflow relations such as prerequisites, data flow, and ordering. On a set of 117 realistic, non-echoing queries, the hybrid ranker retrieves the correct skill within the top five in 73.5% +/- 8.0 of cases, leaving roughly a quarter of queries unserved. When used as the design intended (substituting graph neighbours for additional ranked results at matched
token budget), the graph is significantly worse (-11.2 points, p = 0.0007). Its LLM-generated edge layer adds nothing over neighbours obtained free from a local
embedding pass, and 73% of the queries the ranker misses are not reachable through the graph at all. We attribute this to a pre-filter
topology bound. Because the graph's candidate edges are drawn from the same
embedding neighbourhood the ranker already searches, 98.6% of typed edges connect skills the ranker had already surfaced together. The graph can enrich relation semantics but cannot extend
retrieval reach. We further show that evaluating on author-written queries overstates hit@5 by up to 44 points, which would have hidden these results entirely. Our contribution is a mechanistic account of why added structure does not improve
retrieval over a strong ranker, and identify the conditions under which adding structural interdependence into the
retrieval is optimal.
Read the full abstract ▾
Artificial Intelligence
MicroEvo: Knowledge-Guided LLM Sampling for Efficient Microarchitecture Design Space Exploration
Jia Xiong, Runkai Li, Chenxu Niu et al. (+11 more)
Microarchitecture design space exploration suffers from expansive search spaces and expensive PPA evaluation, leaving only a small simulation budget for design decision-making. Existing methods perform blind search without considering microarchitectural dependencies and fail to learn from the iterative search effectively, leading to wasted evaluations and weak Pareto convergence. In this paper, we propose MicroEvo, a knowledge-guided framework that couples off-the-shelf LLMs with Monte Carlo Tree Search (MCTS) for multi-objective microarchitecture optimization. MicroEvo combines LLM-driven evolutionary operators, a Pareto-aware tree policy that balances Pareto contribution and diversity, an active knowledge accumulation mechanism that extracts and reuses optimization insights, and state-aware directives that adapt the search behavior online. Experiments show that MicroEvo improves Pareto-front quality by up to 36.2% over NSGA-II and achieves 10.6x higher search efficiency, and also demonstrates strong scalability to a complex industrial-scale core. The code repository is available at: https://github.com/GEAR-SEU/MicroEvo-ICCAD-26.
Read the full abstract ▾
All authors Jia Xiong, Runkai Li, Chenxu Niu, Guangyuan Gao, Changwen Xing, Yifan Zhang, Xinlai Wan, Jieran Cui, Chen Bai, Yusheng Hua, Ying Wang, Ming Ling, Xi Wang, Tao Xie
Artificial Intelligence
Schema-Guided Hierarchical Information Extraction and Semantic Evaluation Using Generative AI
Modhurita Mitra, Jan-Willem Versteeg, Maarten D. Schermer et al. (+3 more)
We present a schema-based framework for extracting complex, structured information from unstructured text documents using generative AI, followed by automated semantic evaluation of the extracted information against a gold standard. The schema, serving as an information model encoding domain knowledge, provides a unified, systematic, and consistent framework for extraction of hierarchical, nested information, with attributes of variable cardinality, and subsequent evaluation of the results. Information extraction from a document is performed in a single call to the model, in zero-shot mode. In the evaluation step, we introduce a path-based semantic matching
algorithm to align the nested, variable-cardinality attributes in the extracted results with those in the gold standard. We use generative AI for semantic comparison of the extracted and gold standard values of an attribute, and introduce a rubric to classify the result of the comparison, according to domain-specific considerations, as an exact, semantic, useful, or non-match. We were able to extract 12 out of 14 attributes with an F1 score of $>$90% from documents published by the health technology assessment organisation NICE, using the generative AI model Claude Opus 3. The time needed to extract the attributes from a document was $\sim$30 times lower than the time taken by a human domain expert. We further demonstrate generalisability of this framework across different generative AI models and transferability across different HTA organisations and
languages.
Read the full abstract ▾
All authors Modhurita Mitra, Jan-Willem Versteeg, Maarten D. Schermer, Shiva Nadi Najafabadi, Marie L. De Bruin, Lourens T. Bloem
Artificial Intelligence · Cross-listed from cs.SD
Audio-to-Score Transcription using Pre-trained Features, Data Augmentation, and the New SheetSage-A2S Dataset
Eoin Cummins, Zhongyi Huang, Alexandre D'Hooge et al. (+2 more)
Existing audio-to-score (A2S) systems primarily focus on classical music, and the application to popular music remains underexplored. This paper first presents the new SheetSage-A2S
Dataset, which includes 61 hours of audio with **kern score encodings for 9,468 clips originating from 6,066 unique songs, the first of its kind to facilitate A2S research for popular music. Additionally, we improve on existing A2S approaches by using data augmentation and MuQ, a pretrained feature-extraction model for music audio, to enhance generalisation abilities and extract meaningful audio features. Results show that the proposed A2S model achieves 4.98% symbol error rate (SER) on the Quartets collection for classical music, which significantly outperforms the 15.3% SER from the existing state-of-the-art (Alfaro-Contreras et al. 2024). Additionally, our model achieves 20.92% SER on the SheetSage-A2S
dataset for popular music, serving as a strong
benchmark for future research. The
dataset, model, and code are made publicly available at: https://github.com/
Multimodal-Music-Research-Lab/SheetSage2Kern_model.
Read the full abstract ▾
All authors Eoin Cummins, Zhongyi Huang, Alexandre D'Hooge, Zhuoru Mo, Yaolong Ju
Artificial Intelligence
iARCS: Iterative Agentic RL for Controllable 3D Scene Generation
Saugat Adhikari, Ashok Prasad Neupane, Pramish Paudel et al. (+2 more)
Synthetic 3D scene generation is increasingly used as a data source for computer
vision and
embodied AI, but existing generators often optimize perceptual realism without reliably satisfying task-critical functional constraints. This mismatch limits the usefulness of synthetic data for downstream training, where accessibility, traversability, and spatial rule compliance are often essential. We present iARCS, an iterative agentic reinforcement learning framework that adapts a pretrained scene generator to naturallanguage task requirements. iARCS uses a two-phase strategy: universal-reward pretraining to improve physical plausibility and layout quality, followed by task-specific finetuning with LLM-generated reward programs that are iteratively refined from training feedback. Experiments show improved constraint fidelity on walkability, reachability, and clearance-focused tasks, effective task-specific constraint optimization, and competitive scene diversity. We further show that data generated by iARCS improves a base generator, supporting its value as a practical synthetic data generation tool rather than only a controllable scene editing method.
Read the full abstract ▾
All authors Saugat Adhikari, Ashok Prasad Neupane, Pramish Paudel, Ajad Chhatkuli, Danda Pani Paudel
Artificial Intelligence · Cross-listed from cs.RO
Visual Grounding in Zero-Shot Vision-Language Control
J. de Curtò, Dayani Plasencia, Diego Sánchez et al. (+1 more)
Vision-
language models (VLMs) are increasingly used as zero-shot controllers, but successful trajectories do not necessarily show that decisions are grounded in visual input: simulator dynamics and conservative action priors can produce favourable scores without meaningful perception. We investigate this with an input-ablation battery: blind-image controls, repeated identical inputs, lane-axis reflection, non-visual baselines, and pipeline-integrity checks. Across nine direct-action models, six structured local VLMs, and an exploratory VLM-MPC hierarchy, we analyse 32,874 scored calls over two embodiments and three simulators. The direct-control results are largely negative: a constant-SLOW policy outperforms a scripted
geometric controller, several models are image-invariant or nearly constant, and models that recognize longitudinal hazards still fail to transform LEFT and RIGHT under reflection. No local VLM meets the joint longitudinal and lateral grounding criteria. However, an image-only deterministic positive control estimates the lead gap with 0.090 m MAE and exact mirror equivariance, confirming the stimuli carry sufficient visual information; the failures are modular, not universal. A post-hoc, leakage-controlled
symmetry-consensus guardian selects two models from 16 calibration frames and freezes a 2-of-4 hazard vote across original and reflected views. On 272 held-out frames it reaches 0.954 balanced accuracy (episode-cluster bootstrap 95% CI [0.895,0.990]); nested leave-one-episode-out recovers the same pair and threshold in all 12 folds. Abstaining on ties raises committed balanced accuracy to 0.973 at 0.824 coverage. With deterministic perception retaining lateral authority, offline modular replay achieves 0.934 action agreement and exact mirror equivariance. These results support current VLMs as bounded, selective hazard assistants, not monolithic zero-shot controllers.
Read the full abstract ▾
All authors J. de Curtò, Dayani Plasencia, Diego Sánchez, I. de Zarzà
Artificial Intelligence · Cross-listed from cs.SE
Learning Globally Reusable Skills for Coding Agents
Chen Yang, Jiashuo Tian, Ziqi Wang et al. (+3 more)
Automated skill
evolution enables Large
Language Model (LLM)
agents to continuously improve without expensive retraining. However, existing approaches typically treat skill
evolution as a sequence of local updates, overlooking relationships among skills and often producing overfitted skill updates that fail to generalize across tasks. We propose GSE, a globalized skill
evolution framework that jointly optimizes skill compatibility and skill generalization. To preserve consistency across the skill bank, GSE maintains a Skill Relation Graph (SRG) that explicitly models and co-evolves inter-skill relationships. To improve generalization, GSE performs cluster-based skill consolidation to abstract reusable capabilities from local updates and employs replay-driven verification to prevent overfitting and behavioral regressions. We evaluate GSE on two representative software engineering tasks: bug-revealing test generation and false-positive bug report filtering. Across two state-of-the-art coding
agents, OpenHands and mini-SWE-
agent, GSE consistently achieves the best precision, recall, and F1-score. Compared with existing
evolution techniques, GSE improves precision and recall by 6.1%~34.1% and 31.8%~180.0% for test generation, and by 15.4%~96.4% and 13.1%~19.8% for false-positive filtering. Deployment on an internal industrial
agent further yields a 61.4% improvement in F1-score, demonstrating the effectiveness and generalizability of GSE for evolving effective skills.
Read the full abstract ▾
All authors Chen Yang, Jiashuo Tian, Ziqi Wang, Xinyin Liu, Meiru Ye, Junjie Chen
Artificial Intelligence · Cross-listed from cs.HC
Reducing belief in conspiracy theories as they unfold using large language models
Thomas H. Costello, Nathaniel Rabb, Michael Nicholas Stagnaro et al. (+2 more)
The emergence of conspiracy theories in the wake of major events is a significant societal challenge. Here we test whether conversational dialogues with a large
language model (LLM) can reduce belief in immediately unfolding conspiracies. In experiments conducted in the days following the July 2024 assassination attempt on Donald Trump and the September 2025 assassination of Charlie Kirk, U.S. adults (Experiment 1: N = 472; Experiment 2: N = 1042) holding conspiratorial views about the crisis event engaged in a multi-turn conversation with an LLM prompted to reduce their conspiracy belief. Compared to control participants who either discussed an irrelevant topic with an LLM or viewed a static fact sheet, participants in the LLM treatment showed significantly reduced conspiracy beliefs in both experiments. We also found some suggestive evidence of downstream effects of the LLM treatment, observing reduced belief in some – but not all – analyses of different conspiratorial beliefs one to two months later. These results shed light on the psychology of emerging conspiracies and highlight the potential for scalable, cognitively-focused interventions to counteract misinformation in the immediate aftermath of high-profile societal events.
Read the full abstract ▾
All authors Thomas H. Costello, Nathaniel Rabb, Michael Nicholas Stagnaro, Gordon Pennycook, David Rand
Artificial Intelligence
CogVis: Must Open-Vocabulary Change Detection Perceive the Scene Anew for Every Query?
Zijie Wang, Chen Zhong and Wei He
Earth-surface monitoring requires change detection models capable of recognizing arbitrary semantic categories. Open-Vocabulary Change Detection (OVCD) addresses this need. However, existing methods often entangle temporal perception, semantic discrimination, and region verification, causing unstable results and redundant computation. Inspired by human visual change perception, we propose CogVis, a cognitive
memory-guided framework that reformulates OVCD as a perception-
memory-verification paradigm. CogVis first employs a Scene Change Perceptron (SCP) to extract a reusable, category-agnostic change prior from frozen bi-temporal features, thereby decoupling temporal evidence from semantic category decisions. A Semantic
Memory Calibrator (SMC) then compensates for category-dependent score shifts by dynamically estimating an image-query-specific decision threshold. Finally, an Adaptive Region Filter (ARF) filters connected candidates using learned semantic, temporal, and structural reliability. Experiments on seven
benchmarks spanning semantic change detection, binary change localization, and building-damage assessment show that CogVis achieves state-of-the-art performance across all evaluated
datasets. By sharing scene-level change perception, CogVis further avoids repeating category-agnostic temporal perception across queries and improves inference throughput by 28.50%.
Read the full abstract ▾
Artificial Intelligence
PaDoc: Layout-Grounded Parallel Decoding for Document Parsing
Hao Yu, Jiabo Zhan, Kang Liu et al. (+8 more)
End-to-end document parsers provide a unified interface, but serialize page layouts and regional contents into one autoregressive sequence. This formulation forces independent regions onto a decoding path whose length grows with the total content, whereas crop-based two-stage parsers expose region-level parallelism at the cost of repeated visual prefills and fragmented page context. To retain full-page context while removing dependencies, we propose PaDoc, a layout-grounded parser that treats the predicted layout as a branching structure over a shared page representation. Under a region-sufficiency assumption, we derive a prefix-conditioned factorization in which the layout stream and regional content branches advance concurrently, reducing the decoding depth to the longest layout-content path. We realize this factorization within a single MLLM: packed variable-length ancestor
attention preserves the visibility under standard next-
token training, while masked parallel decoding creates branches that the evaluated vLLM backend serves as concurrent requests with cache-resident shared-prefix reuse. On OmniDocBench Full, PaDoc attains an Overall layout F1 of 91.1 and, among end-to-end parsers, a top-tier Overall score of 94.24 together with the best Text Edit (0.038) and Formula CDM (95.59). On a 384-page subset and one A800 GPU, it is the fastest end-to-end parser at five concurrency levels, improving valid-page throughput by 67.4-118% and reducing P95 latency by 39.2-54.9% relative to a same-backbone Sequential SFT baseline. Code is available at https://github.com/Longin-Yu/Padoc
Read the full abstract ▾
All authors Hao Yu, Jiabo Zhan, Kang Liu, Linnan Zhao, Dongxu Yue, Rui Chen, Jinglin Wang, Chong Sun, Chen Li, Jing Lyu, Chun Yuan
Artificial Intelligence
FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows
Bo Deng, Kang Zhou, Lifan Guo et al. (+6 more)
Agents used over time encounter recurring professional work: each case requires different evidence and judgment, while the underlying workflow can be reused.
Benchmarks built from independent tasks cannot reveal whether an
agent turns earlier experience into better procedures for later cases. We introduce FinEvo-Bench, a longitudinal
benchmark designed around this structure. It contains 120 open-ended tasks drawn from real cases across 20 business scenes in six financial domains. Each scene contains six substantively different cases that share a professional workflow and an expert-authored rubric for task quality and financial compliance. Constructing and validating the
benchmark required approximately 1,200 person-hours. Finance provides a natural test bed because recurring analyses apply shared professional and compliance requirements to heterogeneous inputs, producing case-specific analyses and conclusions. We evaluate four self-evolving
agent scaffolds with Qwen3.7-Max on three independently shuffled, globally interleaved task streams. A Claude Code rubric judge backed by Claude Opus~4.6 evaluates all outputs, and paired state-reset controls estimate each scaffold's gain from retained experience. Evolving runs score 9.33–19.37 points higher and trigger 0.12–0.44 fewer compliance issues per task than their paired controls. Paired score gains at within-scene ranks~4–6 exceed those at ranks~1–3 by 6.10–8.70 points. FinEvo-Bench measures whether retained experience improves later professional work under continued use.
Read the full abstract ▾
All authors Bo Deng, Kang Zhou, Lifan Guo, Chongyang Tao, Xuanren Chen, Chenggang Xie, Renzhao Liang, Feng Chen, Chi Zhang
Artificial Intelligence · Cross-listed from cs.CR
Hardware Keystores for AI Agent Signing Workflows: A Zero-Trust MCP Enforcement Architecture
Leo Sambrook and Sampo Sovio
AI
agents increasingly sign Git commits, certify documents, and attest release artifacts on behalf of their operators, using private keys that live in software-accessible locations (plaintext files, environment variables, container
memory) readable by any process the
agent can reach. A widely deployed
agent framework recently leaked its keys this way to a single email injection. Hardware keystores (HSM, TPM, smart card) keep the key on-device, but exposing the keystore as a tool an LLM
agent can call moves the problem rather than removing it: once a signing session exists, the hardware cannot tell a request reflecting the operator's intent from one injected into content the
agent read. We characterize this confused-deputy problem and build the five-layer Zero-Trust enforcement stack it requires, so that only requests consistent with the operator's committed intent reach the hardware. We evaluate on two attack planes.
Prompt injection in content the
agent reads (AgentDojo, three injection-following models, n=144) falls from an 18.1% baseline attack success rate to 0% under the full stack. Tool poisoning by a compromised MCP server (MCPTox) is contained identically: a hash comparison protects a pre-committed payload, and human-in-the-loop escalation contains autonomous requests with nothing pre-committed. A further probe delineates how far the semantic filter's protection extends: it detects a substitute document under an unrelated name, but an adversarially plausible substitute name defeats it in every trial we ran. We report this as a central finding: the architecture's guarantee never rests on the filter being right, only on a human being asked whenever nothing was committed in advance. The trade-off we characterize across both planes is that the less an operator can commit to in advance, the less deterministic the resulting guarantee, down to asking a human.
Read the full abstract ▾
Artificial Intelligence
Contextual Information Policy Optimization for Search Agents
Xingyu Guo, Wei Chen, Linlin Yang et al. (+1 more)
Search
agents extend large
language models beyond static parametric
memory by enabling them to acquire and use external evidence during multi-step
reasoning. For knowledge-intensive tasks involving complex or evolving information, their reliability depends not only on retrieving relevant evidence but also on using it to guide subsequent
reasoning. However, existing methods primarily reward final-answer correctness or intermediate progress, without directly assessing whether post-
retrieval actions are grounded in the retrieved evidence. This misalignment encourages prior-driven
reasoning:
agents form conclusions based on internal knowledge and use
retrieval mainly to confirm them, resulting in confirmation bias and inefficient evidence use. To address this issue, we propose Contextual Information Policy Optimization (CIPO), an evidence-oriented reinforcement learning framework that explicitly aligns policy optimization with external evidence use. CIPO assigns dense, turn-level credit to
reasoning actions influenced by retrieved information, while combining this evidence-use signal with a global outcome reward to preserve answer correctness. With this manner, CIPO discourages evidence-detached guesses and promotes
reasoning trajectories in which retrieved facts can guide or revise subsequent
reasoning. Importantly, CIPO requires neither human process annotations nor an additional reward model. Extensive experiments on seven in-domain and out-of-domain
benchmarks show that CIPO reduces the prevalence of prior-driven
reasoning and achieves excellent performance on most tasks.
Read the full abstract ▾
All authors Xingyu Guo, Wei Chen, Linlin Yang, Baochang Zhang
Artificial Intelligence
Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts
Massi-Nissa Abboud, Aladin Djuhera, Elena Cabrio et al. (+1 more)
Measuring political bias in large
language models (LLMs) remains challenging as it can manifest through subtle differences in framing, argumentation, and legal
reasoning that are difficult to capture with a single metric. In this work, we introduce POLI-BIAS, a counterfactual framework for measuring whether LLMs treat legally equivalent conflict scenarios differently depending on the countries involved. POLI-BIAS compares responses to paired
prompts in which country identities are systematically swapped across diverse geopolitical relationships, legal violations, and
reasoning tasks. Rather than reducing bias to a single judgment, our framework decomposes response disparities into five interpretable dimensions, revealing how and where unequal treatment manifests. Across 5,600
prompt pairs and seven open-weight LLMs from diverse model families and sizes, we find that models treat countries in active conflict or tension less evenly than fictional or uncontested countries, describing, evaluating, and defending equivalent actions differently. Our results thus establish POLI-BIAS as a fine-grained framework for
auditing political even-handedness in LLMs.
Read the full abstract ▾
All authors Massi-Nissa Abboud, Aladin Djuhera, Elena Cabrio, Holger Boche
Artificial Intelligence · Cross-listed from cs.LG
Is Self-Pretraining really useful to improve diagnosis in medical Time Series?
Omar Coser, Antonio Orvieto, Paolo Soda et al. (+1 more)
Inspired by recent evidence that
transformer architectures benefit from Self-PreTraining (SPT) on long-context
benchmarks, we investigate whether similar gains extend to
multimodal, multivariate, and even simple univariate medical time series. Our objective is to assess the impact of SPT on the performance and scalability of
transformer-based models across diverse medical applications, particularly under limited data conditions. We evaluate
transformer architectures on three representative medical time-series tasks: rehabilitation robotics (Camargo
dataset), stress detection (Non-
EEG Stress), and Parkinson's disease detection (Gait Parkinson's Disease). Models are trained either from scratch or through SPT using four masking-based objectives designed to promote temporal and cross-modal representation learning, and we systematically vary model depth to examine how capacity interacts with pre-training benefits. Across
datasets and configurations, SPT consistently improves classification accuracy by 0-6 percentage points depending on masking strategy,
dataset and architecture, with gains observed not only in multivariate settings but also when models are restricted to simple univariate inputs. The improvements increase for deeper models that can better exploit the enriched temporal representations learned during pre-training. These findings indicate that SPT is a simple and general strategy that enhances
transformer performance on medical time-series tasks without requiring task-specific architectural changes, supporting its potential to improve robustness and accuracy in data-limited clinical settings.
Read the full abstract ▾
All authors Omar Coser, Antonio Orvieto, Paolo Soda, Loredana Zollo
Artificial Intelligence
Mind the Gaps: Mixture-of-Minds for Human Simulation
Pranav Dahiya
Predicting how a population will answer a new question is a long-standing goal. Statistical methods succeed at the level of the mass but falter at the level of the individual. Large
language model simulators inherit this gap. They recover a population's central tendencies while flattening its heterogeneity, and they carry social biases and
prompt brittleness that distort individual predictions. This paper introduces Anacreon, an audience simulation model that targets the individual level within a narrow, well-specified domain. Anacreon learns an authorship
embedding that separates individuals, clusters a real qualitative corpus around seed people, and trains a dedicated adapter for each cluster, a mixture of minds, on a Gemma~4 12B base. It harvests demographics, psychological traits, and survey responses from public text, and augments each record with a chain-of-
emotion. It reduces
prompt brittleness by shuffling response options and reduces positive bias by balancing the training distribution. On a large, externally sourced survey, Anacreon reaches a state-of-the-art ordinal
alignment of 0.775, the individual-level accuracy measure on which the field has converged, with a small residual bias. The work is a step toward drawing aggregate insight from faithfully simulated individuals.
Read the full abstract ▾