Artificial Intelligence · Cross-listed from cs.CV
MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities
Mingqiao Ye, Zhaochong An, Zhitong Gao et al. (+11 more)
Any-to-any models predict any modality from any combination of others within a single network, a formulation used in
multimodal vision and
vision-
language models, and increasingly in scientific domains such as ecology and astronomy. Existing any-to-any models are typically trained from scratch using encoder-decoder or
diffusion architectures, impacting their performance and preventing them from using strong pre-trained decoder-only models as a prior. In this work, we investigate decoder-only any-to-any
multimodal modeling, which treats all modalities symmetrically and supports arbitrary modalities as inputs and outputs without modality-specific heads, losses, or task pipelines. Because every modality is both an input and an output of the same model, the resulting model, named Modus, can support a range of applications, such as chained generation through intermediate modalities or cross-modal self-verification by scoring the model's own outputs with another generated modality. Modus demonstrates strong out-of-the-box performance and is competitive with specialist and multitask baselines using a single model across various
benchmarks. All materials are open-sourced at https://modus-
multimodal.epfl.ch/.
Read the full abstract ▾
All authors Mingqiao Ye, Zhaochong An, Zhitong Gao, Xian Liu, François Fleuret, Chuan Li, Amir Zadeh, Serge Belongie, Afshin Dehghan, Jesse Allardice, David Mizrahi, Oğuzhan Fatih Kar, Roman Bachmann, Amir Zamir
Artificial Intelligence
A Cost-Effective Multimodal LLM Reasoning Framework for Question Answering over Irregular Clinical Time Series
Frank Nie, Ethan B Liu, Yuan Zhu et al. (+2 more)
Question answering (QA) over irregular clinical time series (ICTS) plays a pivotal role in a wide range of healthcare applications. Although recent
multimodal time-series large
language models (LLMs) have shown considerable promise in general-purpose time-series QA, they remain poorly equipped to model the sparsity, asynchrony, and irregular sampling patterns of clinical observations. To fill this gap, we propose ClinPRISM, a cost-effective
multimodal LLM
reasoning framework for question answering over ICTS data. First, we devise an irregularity-aware multi-scale encoder to capture sparse clinical evidence at diverse temporal scales. Then, we propose a temporal evidence distiller to integrate representations across these scales and compress them into a small number of LLM-compatible
tokens. Moreover, we introduce a progressive
alignment strategy that sequentially aligns the irregular trajectories with the LLM's textual
embedding space. To facilitate training, we construct 30,000 clinical time series paired with multi-scale descriptions, together with 41,000 instruction-tuning instances spanning 11 tasks. Using a 4-billion-parameter LLM backbone, ClinPRISM achieves state-of-the-art performance on the held-out evaluation
benchmark while using only 16 time-series
tokens and achieving an average inference latency of 0.15 seconds per question.
Read the full abstract ▾
All authors Frank Nie, Ethan B Liu, Yuan Zhu, Wei Fan, Jindong Han
Artificial Intelligence · Cross-listed from cs.CL
Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases
Rui Yang, Weihao Xuan, Yi Lin et al. (+23 more)
Clinical diagnostic evaluation should not only assess whether models can provide correct diagnoses, but also reflect the realities of clinical practice, including progressive disclosure of
multimodal information, dynamic updating of diagnostic hypotheses, and continuous refinement of clinical
reasoning. However, existing evaluations of
multimodal large
language models (MLLMs) typically rely on single-turn or isolated tasks, making it difficult to fully capture the complexity of real-world clinical diagnosis. To bridge this gap, we developed ClinMM-Bench, the largest multi-turn
multimodal clinical diagnostic evaluation
benchmark to date. ClinMM-Bench contains 1,089 challenging real-world clinical cases and 3,760 medical images across eight specialties. We systematically evaluated 15 representative MLLMs using a two-level evaluation framework that assessed both diagnostic accuracy and diagnostic
reasoning quality. Results showed that proprietary models achieved the highest overall diagnostic accuracy, but the proportion of completely correct diagnoses remained limited across all models. In terms of diagnostic
reasoning quality, current models can identify plausible diagnostic directions but still have considerable limitations in generating reliable diagnostic
reasoning. Error analysis further identified five representative failure modes: information synthesis failure, knowledge mapping error, perception error, premature closure, and visual
hallucination.
Read the full abstract ▾
All authors Rui Yang, Weihao Xuan, Yi Lin, Zhuhan Bao, Jonathan Chong Kai Liew, Matthew Yu Heng Wong, Nicolás Lescano, Nikita R. Paripati, Emily Ling-Lin Pai, Jiarui Liu, Heli Qi, Heng-Jui Chang, Benny Kai Guo Loo, Huitao Li, Kunyu Yu, Yufan Wang, Chuan Hong, Shijian Lu, Douglas Teodoro, Naoto Yokoya, Ross Koppel, Mona Diab, Hua Xu, David W. Bates, Nan Liu, Yifan Peng
Artificial Intelligence · Cross-listed from cs.CV
Face De-Identification: A Domain-Centric Survey from Capture to Processing
Hui Wei, Hao Yu and Guoying Zhao
Face de-identification (De-ID) aims to remove or conceal personally identifiable facial features in images or
videos to prevent identity recognition while preserving utility for downstream tasks. With the rising emphasis on data
privacy and responsible AI, face De-ID has emerged as an active research area spanning computer
vision and
privacy-preserving communities. Early approaches, and many contemporary ones, operate in the digital domain by modifying pixel-level or appearance-level features through post-capture processing. Recent advances extend face De-ID beyond post-processing by integrating
privacy mechanisms directly into sensors during image acquisition, bridging sensing systems and downstream
vision algorithms. In parallel, physical-domain methods explore wearable accessories and materials that conceal identity information in real-world environments prior to capture. In this survey, we present the first unified overview that spans the full data acquisition pipeline, encompassing the physical, sensor, and digital domains. Through this domain-centric lens, we systematically analyze current methodologies, technical progress, and the distinct challenges inherent to each stage. We then review and organize existing evaluation protocols, examining current practices and highlighting the critical need for standardized, comprehensive
benchmarks. Finally, we identify key open problems and outline emerging research directions to guide future work in this rapidly evolving field. To support ongoing research, we maintain a project page that organizes relevant literature with collected
datasets and open source code: https://github.com/CV-AC/Awesome-FaceDe-ID.
Read the full abstract ▾
Artificial Intelligence
dtControl2+$\varepsilon$: Trading Optimality for Explainability in MDPs via Decision Trees
Tereza Kinská, Jan Křetínský, Tobias Meggendorfer et al. (+2 more)
Over the past decade, decision trees have been used to represent controllers (a.k.a. policies) in an explainable way, with dtControl2 as a current state-of-the-art tool. However, for systems that are large or have many corner cases, even such representations tend to be too complex and not human-comprehensible. Unfortunately, reducing the size of the decision tree is not straightforward, as missing just a single crucial case might result in an incorrect controller. We tackle this issue in the setting of Markov decision processes, extending dtControl2 by "$\varepsilon$" functionality: Given an allowed imprecision $\varepsilon \geq 0$, we construct a smaller decision tree, distilling the essence of the controller, while still guaranteeing its $\varepsilon$-optimality. This enables us to provide tunably simpler explanations, omitting a controllable amount of detail. Our tool constructs decision trees that are orders of magnitude smaller than the state of the art.
Read the full abstract ▾
All authors Tereza Kinská, Jan Křetínský, Tobias Meggendorfer, Sabine Rieder, Maximilian Weininger
Artificial Intelligence · Cross-listed from cs.CV
Evaluating VLMs for Autonomous Agent-Driven Geometry Clipping Detection in Video Game QA
Carlos Celemin, Benedict Wilkins, Adrián Barahona-Ríos et al. (+2 more)
In this work, we study the use of
Vision-
Language Models (VLMs) for anomaly detection in an
agent-driven game Quality Assurance (QA) pipeline focusing on geometry clipping. In this evaluation, a custom exploration
agent navigates a game level to collect visual observations, while the automatic annotation pipeline provides frame-level clipping labels. This setup allows us to evaluate recent VLMs on a controlled anomaly detection task without manual annotation. We
benchmark six recent VLMs (Gemini, GPT, Qwen, Gemma, Llama, and Ministral) under a zero-shot prompting setting and analyse their sensitivity to four
prompt variants. Our results show that while the VLMs can capture visual cues associated with geometry clipping, they all produce substantial false positives on visually ambiguous frames such as near-contact geometry and partial occlusions. Gemini-3.1-Flash achieves the best overall accuracy and is the most robust to
prompt variation, while open-source models exhibit large precision–recall swings depending on the
prompt design. These findings suggest that current VLMs are best suited as high-recall candidate filters within multi-stage QA pipelines rather than as standalone bug detectors.
Read the full abstract ▾
All authors Carlos Celemin, Benedict Wilkins, Adrián Barahona-Ríos, Saman Zadtootaghaj, Nabajeet Barman
Artificial Intelligence
Penelope: Localized Latent Recurrence for Efficient Structured Reasoning
Yutong Chen, Shouqian Shi, Xinran Liu et al. (+5 more)
Complex structured
reasoning tasks often require additional computation, yet current
language models obtain it mainly by increasing parameter scale or by serializing intermediate steps as chain-of-thought (CoT)
tokens. The former raises training and deployment costs, while the latter ties
reasoning computation to autoregressive output length. We introduce Penelope, an efficient latent-
reasoning framework for pretrained decoder-only
Transformers that localizes recurrent computation to a selected decoder interval. The lower decoder prefix is evaluated once to construct a problem-conditioned boundary
memory, which is then iteratively refined through time-modulated GRU dynamics and recurrent readout states before answer generation. A progressive CoT-to-latent curriculum transfers visible
reasoning into this internal recurrent path, allowing additional computation to be allocated in latent space without repeatedly executing the complete decoder or generating a long intermediate trace. Experiments on open-source structured-
reasoning benchmarks show that, at validation-selected latent budgets, Penelope attains competitive accuracy relative to established latent-
reasoning models while reducing measured inference latency. These results show that latent refinement can be localized to a narrow decoder interval, reducing repeated full-decoder execution without generating a long visible
reasoning trace and providing a practical accuracy-efficiency tradeoff for decoder-only
Transformer models.
Read the full abstract ▾
All authors Yutong Chen, Shouqian Shi, Xinran Liu, Haochen Wang, Jiaying Wang, Tianxing Xu, Yuanxi Wang, Zirui Ding
Artificial Intelligence
Toward Standardized Cross-Vendor Agent Tool Trust Management in Autonomous Networks
Ravi Kant Sharma, Ashutosh Uttam and Ajay Kumar
Autonomous Network Levels 4-5 require AI
agents to invoke tools across vendor boundaries without human oversight, yet existing management standards lack a standardized mechanism for cross-vendor trust visibility. When a tool from Vendor B is compromised,
agents from Vendor A continue invoking it – unaware of the trust degradation – causing cascading service impact. We present AgentToolMO, a proposed 3GPP NRM information model for
agent tool trust management. The model comprises: a formally defined trust state machine with provable graduated enforcement, damped cascade propagation with bounded convergence, cross-vendor trust notifications via existing Management Services (MnS) interfaces, and retroactive impact assessment through NRM dependency graph traversal. Simulation-based evaluation across multi-vendor
topologies shows that standardized cross-vendor notifications reduce blast radius from hours-scale undetected propagation to near-real-time containment bounded by MnS notification delivery, with cascade convergence guaranteed in bounded iterations and sub-linear notification scaling across vendor domains. The framework operates within existing 3GPP management infrastructure, leverages existing protocols, and provides a standardization pathway for trustworthy multi-vendor autonomous network management.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
SAM3D-Guided Object-Centric Representation Alignment for Vision-Language-Action Models
Zonghe Liu, Shanyuan Jie, Xiaoquan Sun et al. (+4 more)
Vision-
Language-Action (VLA) models have shown strong potential for general
robot manipulation, but most existing models rely on 2D visual-
language backbones and lack fine-grained 3D understanding of target objects, especially under occlusion, pose variation, scale changes, and precise spatial interaction. We propose an object-centric 3D representation
alignment framework built upon $\pi_0$, using SAM3D as a frozen 3D teacher to provide target-object 3D priors during training. Specifically, we localize task-relevant objects with object recognition models, generate corresponding object masks, and use SAM3D to extract dense object-level 3D representations, which are aligned with intermediate visual features of $\pi_0$. This enables the policy to internalize target-object 3D information while preserving the original RGB-
language-to-action inference pipeline without requiring depth, point clouds, masks, SAM3D, or additional 3D modules at test time. Simulation experiments show consistent improvements, achieving 99.1% on LIBERO and an average length of 4.11 on CALVIN. Real-world experiments further demonstrate that our method is particularly effective in long-horizon manipulation scenarios where the
robot must focus on different target objects across multiple subtasks.
Read the full abstract ▾
All authors Zonghe Liu, Shanyuan Jie, Xiaoquan Sun, Chen Cao, Zetian Xu, Zongsheng Liu, Jiayu Chen
Artificial Intelligence · Cross-listed from cs.HC
AnnoBench: A Benchmark for Visualization Annotation Generation
Md Rahat-uz-Zaman, Md Dilshadur Rahman, Andrew McNutt et al. (+1 more)
Annotation is among the most demanding visualization tasks to automate, as it simultaneously requires correctly navigating visual, semantic, and stylistic constraints. Failure to meet any of these conditions severely undermines the utility of an annotation, rendering it challenging to read, inaccurate, or visually discordant. Despite a growing body of annotation tools and automations, no existing
benchmark or evaluation framework tests whether these conditions are met because of their scope and annotation not being the focus of their studies. We introduce AnnoBench, a
benchmark for visualization annotation that materializes the inherent challenges of this domain in a structured and testable manner. AnnoBench pairs visualizations from professional data journalism and visualization galleries with annotation tasks, spanning four representation formats, five chart description conditions, and two
prompt specification levels. The
benchmark is executed via VLM-as-a-judge, using models aligned with manual human assessment. We evaluate the
benchmark via four one-factor-at-a-time experiments, exploring the effects of input representation, semantic context, and
prompt specificity, and model selection on annotation quality. This work provides a foundation for advancing annotation automation, tooling, and visualization-generation pipelines.
Read the full abstract ▾
All authors Md Rahat-uz-Zaman, Md Dilshadur Rahman, Andrew McNutt, Paul Rosen
Artificial Intelligence · Cross-listed from cs.LG
Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models
Deepanshu Mody, Samarth Agarwal, Utkarsh Mittal et al. (+1 more)
Activation steering controls model behavior by editing internal activations at inference time. We study its input-side dual: optimizing a fluent
prompt so that a chosen internal latent is driven toward zero, with no inference-time model access. Our target is an "evaluation-awareness" latent-linearly readable and steerable in recent work-whose control would threaten the validity of safety evaluations if models behave differently when they detect being tested. Adapting Fluent Dreaming / EPO with a negated feature term (GCG-style
token optimization plus a self-cross-
entropy fluency regularizer, swept over a fluency weight), we suppress the latent under five target constructions-a CAA direction, a subspace norm, an SAE feature, a single MLP
neuron, and a behavioral logit-on Llama-3.2-3B and Llama-3.1-8B. The latent is robustly suppressible ($z\approx-7$), and a causally-validated Llama Scope SAE feature can be fully and selectively turned off. But our controls tell a cautionary story about the CAA direction: a placebo random direction is suppressed just as hard and shifts behavior just as far, and when we hold a real eval passage in context and optimize only a prefix, suppressing the eval-direction fails to reduce-and slightly increases-the model's behavioral eval judgment. Activation-readability, in short, is not behavioral controllability. We further find that a single MLP
neuron is eval-correlated but not causal at both scales, and that scanning the real Pile yields a natural-text baseline competitive with the optimizer for the internal direction. A positive control validates our erasure detector, bounding an erasure-vs-rotation question earlier left open.
Read the full abstract ▾
All authors Deepanshu Mody, Samarth Agarwal, Utkarsh Mittal, Dipesh Mahato
Artificial Intelligence
Interactive Reward Agent: GUI Task Evaluation via Environment-State Verification
Chenrui Shi, Yuwei Wu, Yang Liu et al. (+5 more)
Graphical user interface task evaluation aims to determine whether a GUI
agent has successfully completed a user instruction. Automated GUI task evaluation has received increasing
attention because the evaluation results can serve as reward signals for both test-time scaling and post-training. However, reliable GUI task evaluation remains challenging because the judgments often require access to environment states, such as system configurations, file data, and application settings, beyond the screenshots of execution trajectories. In this paper, we propose an interactive reward
agent (IRA) based on a propose-then-verify framework to acquire and verify evidence from the post-execution environment. Given a task instruction and a GUI environment after the GUI
agent execution, IRA first proposes the task completion conditions and then verifies them by invoking system tools, application tools, and GUI tools. This design combines evidence from both visible interfaces and the environment state in an interactive process. We further introduce GUI-RewardBench, a
benchmark of 321 GUI task trajectories spanning 10 Ubuntu desktop application categories. Experiments show that IRA achieves 86.9% accuracy on GUI-RewardBench, outperforming existing evaluator baselines. We further apply IRA to reinforcement learning of GUI
agents, achieving a 34.0% OSWorld success rate, which demonstrates that IRA can provide effective reward signals for training GUI
agents.
Read the full abstract ▾
All authors Chenrui Shi, Yuwei Wu, Yang Liu, Ruining Feng, Zirui Shang, Zhi Gao, Lifeng Fan, Che Sun
Artificial Intelligence
Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation
Stefan Krsteski, Charlotte Meyer, Guillaume Allegre et al. (+2 more)
Comprehensively evaluating AI
agents across interactive environments is difficult due to fragmented tasks, scaffolds, verifiers, and scoring rules. Unfortunately, existing efforts to unify these evaluations are limited in scale and domain, making costly reruns necessary and leaving available data incomparable. We introduce MESSIER, a unified corpus of 957,611 records spanning 30
benchmarks, 745
agents, 11,891 tasks, and 74,263 verifiers. MESSIER combines public evaluation results with new runs on six underrepresented professional and scientific
benchmarks, standardizing their heterogeneous components into a common schema. Using this corpus, we show that frontier progress is uneven across
benchmark groups, with function-calling evaluations largely saturated, programming improving fastest, and enterprise workflows remaining most challenging. Counterfactual rescoring further shows that strict all-pass scoring in multi-verifier tasks can alter
agent rankings. Finally, we derive capability scores from our corpus that correlate with Epoch's Evaluation Capability Index rankings at Spearman h̊o} = 0.84. The scores can also be estimated for subsets defined by domain, occupation, action space, or verifier type. In essence, MESSIER is a reusable resource for studying
agent performance at scale, and a basis for designing better evaluations.
Read the full abstract ▾
All authors Stefan Krsteski, Charlotte Meyer, Guillaume Allegre, Tony O'Halloran, Alexandre Sallinen
Artificial Intelligence
Distributing Security Controls Through Harness Engineering
William Robert Gore
AI coding
agents are being adopted at historic speed, yet security and risk concerns remain the primary barrier to scaling agentic AI across organizations. Existing security controls for coding
agents are not systematically distributed to engineering teams, and vendor-native solutions introduce ecosystem dependencies that may not suit every deployment context. This paper investigates whether off-the-shelf security controls can be implemented on commercial AI coding
agents and scaled to a distributed user base via a custom
agent harness. A phased testing methodology was applied across four
agent configurations — two commercial
agents with and without controls, a baseline harness, and a security-hardened harness — using a 23-test suite derived from the OWASP Top 10 for Agentic Applications. SHarD (Secure Harness Distribution), a distributable harness built on the Pi
agent harness, demonstrated that three categories of security controls — OS sandboxing, skill scanning, and tool restriction — can be embedded and distributed via a single install command while retaining equivalent efficacy to direct installation on commercial
agents. SHarD achieved an adjusted score of 100%, matching the best securely configured commercial
agent, with no regression across any test category. Notable observations include evidence that model non-determinism produces inconsistent security outcomes and that autonomous
agent behavior can cross system boundaries in ways that OS sandboxing directly mitigates. Initial characteristics toward a control harness fitness framework are proposed, and a third research question is identified for future investigation.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from eess.AS
Depression Markers in Speech: An Approach based on Tract Variables Dynamics
Sahar Altalhi, Tanaya Guha and Alessandro Vinciarelli
This study identifies new depression biomarkers based on the dynamical properties of tract variables, which represent
geometric features describing the configuration of the
speech articulators. A key advantage of this approach lies in its ability to quantify aspects of the articulatory process that have not been previously explored in the context of depression, namely predictability, complexity, and randomness. These properties are respectively characterised using the Largest Lyapunov Exponent, the Correlation Dimension, and the Sample
Entropy. Thorough experiments were conducted on the Androids Corpus, a publicly available
dataset comprising 64 speakers diagnosed with depression by clinicians and 54 control speakers with no reported history of mental health conditions. The results indicate that the proposed biomarkers effectively discriminate between the depressed and control speakers, as evidenced by the high Cliffs delta values across both read and spontaneous
speech.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from eess.AS
Device Invariance using Domain Adaptation on Acoustic Scene Classification
Abhishek dileep, Shubham Sharma and Padmanabhan Rajan
This paper explores the effectiveness of domain adaptation techniques when using convolutional neural network (CNN)-based and
transformer-based feature representations for acoustic scene classification. Two well-known domain adaptation techniques, namely domain
adversarial neural network (also called DANN) and conditional domain
adversarial network (also called CDAN) are evaluated under various domain shifts. Our study indicates that DANN provides effective domain adaptation fairly consistently for both feature extractors. On the other hand, CDAN provides effective domain adaptation only for CNN-based feature extractors. The study gives insights into how domain adaptation methods may need to be tailored to the underlying feature representation. Experimental evaluation with multiple devices on the DCASE 2020
dataset supports the observations.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
A Machine-Learning-Based Gas Lift Optimization Workflow for Unconventional Fields
Sha Miao, Alexandra Vendetti, Logan Smart et al. (+11 more)
In this paper, we present an automated data-driven workflow using Machine Learning (ML) for gas lift optimization in unconventional fields. This workflow integrates a ML model that accurately forecasts the Gas Lift Performance Curve, and a
Bayesian Optimization Framework to solve for the optimal gas injection rates under the constraints of facility capacity. The ML model leverages the historical production time series data without requiring downhole gauges or multi-rate well tests. We piloted this workflow on 30 wells across 5 well pads in Bakken and obtained >5% production uplift on average. With the success of the pilot, we have now fully-deployed this workflow in Bakken across 200+ gas lift and plunger-assisted gas lift (PAGL) wells. Moreover, the ML-based gas lift optimization workflow presented in this paper is an effective and economic solution for other assets where downhole data or multi-rate testing are not available/feasible due to cost or facility constraints.
Read the full abstract ▾
All authors Sha Miao, Alexandra Vendetti, Logan Smart, Gunta Chomchalerm, Yang Chen, Christopher Frazier, Dustin Haralson, Jeremy Sorenson, Xiao Ma, Huafei Sun, Aaron Shinn, Haining Zheng, Xiao-Hui Wu, Peng Xu
Artificial Intelligence · Cross-listed from cs.CR
Stemma: Induced Decision Regions Reveal LLM Provenance
Keyu Zhang, Vadim Safronov and Andrew Martin
LLM provenance testing asks whether a suspect LLM belongs to the same lineage as a source. Existing black-box methods largely infer this relationship from response-level characteristics, but these characteristics may shift under adaptation or deployment even when the underlying meaning remains unchanged, weakening the reliability of provenance evidence. To address this limitation, we introduce induced decision regions by mapping open-ended outputs into a finite decision space, thereby abstracting away surface-form variation and reframing provenance testing as measuring the inheritance of decision regions. Empirical analysis shows that the source's induced regions are preserved more strongly in related models than in unrelated models. Building on this signal, we propose Stemma, a practical black-box LLM fingerprinting method that operationalises stability, robustness, and specificity as complementary probe-selection principles for reliably estimating induced decision region inheritance. Across 770 source-suspect pairs drawn from 56 public checkpoints and spanning diverse model-weight transformations, Stemma achieves 0.967 AUC and 87.8% TPR at 1% FPR, substantially outperforming four representative baselines. It further achieves 0.995 AUC and 93.5% TPR at 1% FPR on 1,260 pairs covering 91 deployment instances, demonstrating robustness to diverse inference-time deployment settings.
Read the full abstract ▾
Artificial Intelligence
Runtime Uncertainty Monitoring for LLM-Based Multi-Agent Systems Using Bayesian Networks
Bart Custers and Koorosh Aslansefat
This paper investigates how multi-
agent systems (MAS)-based on large
language models (LLMs) can support actuarial risk modelling, with a particular focus on uncertainty quantification. Actuarial workflows represent a high-stakes decision-support setting where unreliable outputs may lead to incorrect risk assessment, unfair pricing, and regulatory non-compliance. To address uncertainty introduced by the probabilistic nature of LLMs and dependencies between
agents, a multi-
agent framework is proposed in which specialised
agents perform data preparation, modelling, review, and explanation tasks under a central hub. The main contribution is a novel approach to uncertainty propagation using
token-level log-probabilities and a
Bayesian Network. Importantly, log probabilities are not treated as direct probabilities of correctness or task success. Instead, length-normalised log-probability summaries are transformed into calibrated task-level confidence estimates before incorporation into the
Bayesian Network. Results show that the framework reproduces baseline actuarial performance while providing additional insight into workflow stability and runtime uncertainty propagation.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
A2TTA: Anchored-and-Agile Test-Time Adaptation for Evolving Traffic Sensor Networks
Du Yin, Xiachong Lin, Yue Tan et al. (+4 more)
Traffic
forecasting is important for efficient traffic management and route planning in smart cities. Existing traffic
forecasting studies typically assume fixed sensor graphs, overlooking the continuous
evolution of real-world traffic networks, e.g., ongoing road network construction and evolving human mobility patterns. These dynamic changes can substantially degrade conventional
forecasting models, motivating test-time adaptation (TTA) to efficiently adapt pretrained models during deployment. However, applying TTA to evolving traffic sensor networks remains challenging in two aspects. First,
topology expansion introduces new sensors and connections, continuously reshaping the sensor graph. Second, tem- poral shifts vary in time scale and stability, requiring differentiated adaptation to long-term and short-term shifts. In this study, we address these challenges by proposing A2TTA, an Anchored-and-Agile Test-Time Adaptation framework for evolving traffic sensor networks, which transforms
topology-induced
forecasting errors into an expandable output calibration problem and separates tem- poral adaptation into persistent global correction and agile context-specific specialization. By jointly addressing
topology evolution and multi-scale temporal shifts, A2TTA enables efficient and robust adaptation to continuously evolving traffic environments. Extensive experiments on ten real-world traffic networks demonstrate that A2TTA consistently improves
forecasting performance across different backbones,
datasets, and prediction horizons. Our code is available in https://github.com/lixus7/A2TTA.
Read the full abstract ▾
All authors Du Yin, Xiachong Lin, Yue Tan, Jinliang Deng, Estrid He, Hao Xue, Flora D. Salim
Artificial Intelligence · Cross-listed from cs.SE
How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair
Ramtin Ehsani, Irene Manotas, Saurabh Pujar et al. (+2 more)
Large
Language Model (LLM)-based Automated Program Repair systems are advancing rapidly, yet their performance remains inconsistent. Even when provided with the same contextual information, an LLM may generate a correct patch for one bug but fail on another closely related bug. Why this happens remains poorly understood, and it is unclear how LLMs prioritize the diverse information in bug reports and whether model
attention affects repair success. In this paper, we present the first empirical study of
attention patterns in LLM-based program repair, providing interpretable insights into how models process bug reports and where their
attention is concentrated during repair. We analyze 319 real-world Python and Java bugs from SWE-bench Verified and Multi-SWE-bench to study (RQ1) how model
attention is distributed across bug report sections, (RQ2) how
attention patterns within each section differ between successful and unsuccessful repairs, and (RQ3) how these patterns compare to information developers consider important for bug fixing. We find that successful repairs are characterized by diffused
attention across multiple diagnostic components such as bug descriptions, stacktraces, and test cases, while failures often exhibit over-localized
attention toward metadata such as version information. We further observe that stronger
alignment between model
attention and developer-identified key sections and phrases is associated with higher repair success. Our results provide the first empirical evidence that
attention misallocation is a key factor in LLM-based APR failures, and offer actionable insights for designing more interpretable and reliable future APR systems.
Read the full abstract ▾
All authors Ramtin Ehsani, Irene Manotas, Saurabh Pujar, Luca Buratti, Preetha Chatterjee
Artificial Intelligence · Cross-listed from quant-ph
OmniQEC: discovering practical quantum error-correcting codes by an AI scientist
Ge Yan, Shanchuan Li, Pengyue Ma et al. (+5 more)
Quantum error correction (QEC) is indispensable for scalable fault-tolerant
quantum computing. However, discovering QEC codes that remain effective is challenging, as logical performance depends on the interplay between code structure, hardware, syndrome extraction, and decoding, which often impose competing requirements. Here we introduce OmniQEC, an efficient AI scientist for discovering QEC codes suited to deployment on modern
quantum processors. OmniQEC formulates QEC design as an iterative discovery process in which an orchestrator, implemented by advanced large
language models (LLMs), coordinates code generation, code-level screening, syndrome-extraction synthesis, and decoder-based
circuit evaluation. At its core, OmniQEC combines a self-evolving
reasoning mechanism with a slow–fast synergistic workflow: a fast loop explores candidates using inexpensive code-level proxies, whereas a slow loop performs physically grounded
circuit-level evaluation and feeds the resulting evidence back into the search. We evaluate OmniQEC across four qLDPC construction families, three LLM backends, and $14$ total-physical-
qubit budgets per backend. The discovered codes show steadily improving logical-error suppression with increasing physical-
qubit budgets and outperform the BB codes with $[\![72,12,6]\!]$ and $[\![144,12,12]\!]$ under complete-implementation budgets of 98 and 240 physical
qubits, respectively. The discovered codes are hardware-friendly and may be of independent interest for practical QEC implementation. These findings pave the way towards LLM-assisted QEC discovery grounded in physically informed code–
circuit–decoder co-design.
Read the full abstract ▾
All authors Ge Yan, Shanchuan Li, Pengyue Ma, Qixin Zhang, Pingchuan Ma, Jianping Wang, Min-Hsiu Hsieh, Yuxuan Du
Artificial Intelligence
HiSkill: Empowering LLM Agents with Hierarchical Skill Graphs
Yu Hao, Jinxuan Cai, Qi Zhang et al. (+4 more)
Skills have become an important abstraction for enabling large
language model (LLM)
agents to reuse past experience in long-horizon interactive tasks. However, existing trajectory-to-skill methods often produce flat collections of high-level textual skills that are stored and retrieved independently, leaving skill relations underutilized and maintaining a gap between high-level skills and executable actions. In this paper, we propose HiSkill, a hierarchical skill graph framework that organizes interaction trajectories into a directed graph with skill nodes, AtomicOp nodes, and typed edges. Specifically, the graph connects reusable high-level skills with executable action templates, while also capturing decomposition, temporal transition, compatibility, support, and recovery relations among them. At inference time, HiSkill retrieves a compact task-relevant subgraph and performs subgraph-guided task execution, where a symbolic task state, an active skill, and the retrieved subgraph guide the LLM
agent to switch skills, select AtomicOps, and ground executable actions iteratively. Experiments on three interactive environments show that HiSkill outperforms state-of-the-art baselines while reducing inference
token consumption, demonstrating the effectiveness of bridging high-level skills and executable action grounding through a hierarchical skill graph. Our data and code is available at https://github.com/BUPT-GAMMA/HiSkill.
Read the full abstract ▾
All authors Yu Hao, Jinxuan Cai, Qi Zhang, Yawen Li, Zhiqiang Zhang, Chuan Shi, Cheng Yang
Artificial Intelligence
Distributed Constraint Optimization via Online Learning and Iterative Pricing with Application to Large-Scale Satellite Scheduling
Itai Zilberstein, Pranav Rajbhandari, Steve Chien et al. (+1 more)
Distributed constraint optimization problems (DCOPs) provide a popular framework for distributed decision making under limited communication, but many real-world instances are too large to solve monolithically. We address this challenge from two complementary directions. We revisit the connection between DCOPs and potential games, and adapt modern online learning
algorithms for
equilibrium finding to DCOPs. We show that these
algorithms are competitive with representative incomplete DCOP
algorithms. We then turn to decomposition frameworks for large-scale DCOPs, motivated by large-scale decentralized satellite scheduling. We propose a new framework that separates a DCOP into two interacting subproblems: a high-level meta-DCOP for task allocation, and independent local optimization problems for scheduling. To couple the two levels, we develop a novel iterative pricing method that updates the meta-level utilities using feedback from the local optimizers. Combining our online learning methods with our iterative pricing framework, we obtain near-optimal performance on real-world decentralized satellite scheduling problem instances, fulfilling over 99% of observation requests compared with 87% for state-of-the-art baselines.
Read the full abstract ▾
All authors Itai Zilberstein, Pranav Rajbhandari, Steve Chien, Tuomas Sandholm
Artificial Intelligence · Cross-listed from quant-ph
Lowering the implementation barrier of neutral-atom quantum computing with agentic workflows
Constantin Dalyac, Alexandre Dauphin, Loïc Henriet et al. (+1 more)
Quantum computers are moving from research laboratories to industrial machines accessible via the cloud and integrated into high-performance computing facilities. However, translating theoretical
quantum protocols into hardware experiments remains a major bottleneck, requiring expertise across protocol design, compilation, simulation, and cloud execution. Here, we introduce an agentic workflow that automates this pipeline on neutral-atom
quantum processors (here two Pasqal QPUs available on the cloud) while keeping the researcher in the loop for critical validation. In three case studies from many-body physics and optimization, the
agent went from published paper or patent to a QPU campaign run overnight. In particular, human intervention was crucial to ensure scientific validity: the
agent selected an inadequate observable in one experiment and constructed a plausible but incorrect hardware diagnosis in another, with both failures detected only through domain-expert review. Finally, we use a second
agent to classify a corpus of 633 Rydberg-array arXiv papers and show that nearly half are implementable on present-day QPUs while identifying specific hardware upgrades needed for the rest. Together, these results demonstrate that agentic workflows provide a practical bridge between theoretical ideas and physical hardware, opening
quantum experimentation to a much broader scientific community.
Read the full abstract ▾
All authors Constantin Dalyac, Alexandre Dauphin, Loïc Henriet, Christophe Jurczak
Artificial Intelligence
Speculate While You Reason: Teaching Agents to Predict Their Next Tool Call via Joint Agent-Speculator RL
Jiabao Ji, Yujian Liu, Li An et al. (+4 more)
Large
language model
agents often spend substantial wall-clock time waiting for tool call results. Tool-call speculation can hide this latency by predicting and pre-executing an
agent's next tool call if the prediction matches the
agent's eventual tool call, but existing speculators are typically separate draft models or cached traces that are poorly aligned with the deployed
agent's own behavior. We identify this speculator-
agent gap and show that the target
agent itself is a strong next-call speculator. This points to a simpler design: unifying the
agent and speculator within the same model. In this paper, we introduce the self-speculating
agent, a single model that both solves tasks in
agent mode and predicts its next tool call from partial trajectories in speculator mode, fully reusing prefix KV cache. To enable this dual-mode
agent without degrading performance, we propose a joint
agent-speculator reinforcement learning method, which derives speculation targets from the
agent's own rollouts and alternates
agent and speculator updates. Across agentic search QA and conversational tool-use agentic tasks, our method improves average next tool-call Hit@1 from 44.1 to 61.2 for Qwen3-4B and from 48.9 to 66.3 for Qwen3.5-4B, while preserving
agent task success.
Read the full abstract ▾
All authors Jiabao Ji, Yujian Liu, Li An, Rohit Jain, Gungor Polatkan, Siyu Zhu, Shiyu Chang
Artificial Intelligence · Cross-listed from cs.LG
SpectONet: A Physics-Guided Spectral Deep Operator Network for Euler-Bernoulli Beam Dynamics
Shivani Saini, Ramesh Kumar Vats and Arup Kumar Sahoo
This paper proposes a novel physics-guided spectral deep operator network, termed SpectONet, for solving Euler-Bernoulli beam (EBB) vibration problems. The proposed framework integrates the operator-learning capability of DeepONet with physics-informed constraints and Chebyshev-Gauss-Lobatto (CGL) sensor placement. Unlike conventional DeepONet frameworks, which commonly employ uniformly distributed sensors, SpectONet uses nonuniform spectral sensor locations with a higher concentration of points near the domain boundaries. This sampling strategy improves the finite-dimensional representation of boundary-sensitive structural responses while requiring only a limited number of branch-network inputs. The governing beam equation, together with the associated initial and boundary conditions, is incorporated into the training objective to promote physically consistent and generalizable predictions. Numerical experiments on three EBB vibration problems demonstrate the effectiveness of the proposed framework. Comparisons with strong baseline models such as Vanilla DeepONet, PI-DeepONet, PINN, and CNN-UNet show that SpectONet consistently achieves lower prediction errors across all considered evaluation metrics. In particular, SpectONet achieves an average reduction of approximately 79% in the mean relative $\mathcal{L}_2$ error compared with the best-performing baseline across the three
benchmark problems. These results demonstrate the effectiveness of SpectONet as an accurate physics-guided operator-learning framework for structural vibration analysis.
Read the full abstract ▾
Artificial Intelligence
Loss Invariance Determines What Concept Layers Encode: Volume Grounding in Echocardiography
Hyunkyung Han and Min Jung Kim
Objective: Concept bottleneck models route prediction through interpretable intermediate variables, and their validity is normally judged by how accurately those variables are predicted. We ask whether that judgement is sufficient, using left ventricular volumes as the concepts underlying ejection fraction estimation from echocardiographic
video. Methods: A
video transformer encoder was trained on a publicly available echocardiography
dataset. End-systolic and end-diastolic volumes formed a concept layer from which ejection fraction was computed analytically, with no residual path to the output. We compared training under an ejection fraction objective alone against training with additional supervision of the volumes in millilitres, and evaluated both on 1276 held-out studies. Results: The concept bottleneck did not increase ejection fraction error relative to direct regression, at 6.89 against 7.13 mean absolute error. Without volume supervision, however, the spread of predicted volumes collapsed to 0.1 millilitres against reference spreads of 35.7 and 45.7 millilitres, while correlation was partly preserved. We show that this follows from an invariance property of the objective: ejection fraction is a ratio and is unchanged when both volumes are rescaled, so the loss determines the concept layer only up to scale. Supervision in absolute units reduced volume error from 89.8 to 25.8 millilitres at a cost of 0.4 in ejection fraction error. Conclusion: Concept accuracy alone can conceal a concept layer that carries no physical scale. Significance: Interpretable intermediate variables in clinical models should be validated against the invariance structure of the training objective, not only against prediction accuracy.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Image Quality Dependent Degradation for AI Systems
Yannick Kees, Elena Hoemann, Frank Köster et al. (+1 more)
Perception is one of the primary applications where neural networks outperform conventional
algorithms. One example is AI systems for automated driving, which can detect pedestrians based on image data and avoid them accordingly. A substantial challenge with these AI systems is that their output depends heavily on the quality of the input images. For example, if an image is of inferior quality due to heavy contamination, such as noise or darkness, accurate predictions are hardly feasible. Additionally, various types of errors can occur, each with varying relevance to the trustworthiness of the underlying AI system. In particular, it may be more critical not to detect an existing person than to detect a person where there is none. Therefore, we want to show that we can still avoid the most critical errors in situations of inferior image quality. To achieve this, we aim to establish a fail-degraded system by lowering the network's confidence threshold based on the estimated image quality, enabling it to detect objects more cautiously in uncertain situations. Additionally, we present a novel method for estimating the quality of incoming images by comparing them to the training data using normalizing flows. We will also conduct experiments applying our method to state-of-the-art object detection. In summary, we will present a design strategy for AI-based systems in automated driving that can deal with poor-quality input data without resorting to fallback solutions. Such measures enhance trust in AI-based systems and lead to an increased provision of the AI component.
Read the full abstract ▾
All authors Yannick Kees, Elena Hoemann, Frank Köster, Sven Hallerbach
Artificial Intelligence · Cross-listed from cs.RO
Shared Voxel-Map-Based Cooperative Indoor UAV Guidance with a Multi-Agent Soft Actor-Critic Controller
Thomas Hickling, Dylan Wynne, Yu Su et al. (+1 more)
This paper presents a cooperative indoor
UAV guidance framework that combines a shared voxel-map world model with a multi-
agent Soft Actor-Critic (MASAC) controller. Multiple drones fuse 360
LiDAR observations into a common world-frame occupancy map, which is converted into a compact bird's-eye-view (BEV) representation and provided to each
agent as an ego-aligned local crop. This integrate-in-world, act-in- ego design enables consistent multi-
UAV spatial fusion whilst retaining decentralised continuous control. The policy combines BEV map features, near-field obstacle observations, and compact goal and peer-state information within a centralised-training, decentralised-execution framework. In simulation, the learned controller achieves a 90.3% success rate in corridor
navigation, outperforming Astar planning, an artificial potential field controller, and a prior guidance method. To address residual sim-to-real mismatch, the simulation-trained policy is further adapted using offline imitation fine-tuning from real-world data. Real-world experiments in GNSS-denied indoor environments demonstrate stable two-
UAV cooperative operation across increasingly chal- lenging obstacle layouts. The results show that shared voxel-map representations provide an effective and scalable spatial substrate for learned cooperative indoor
UAV guidance.
Read the full abstract ▾
All authors Thomas Hickling, Dylan Wynne, Yu Su, Nabil Aouf