Artificial Intelligence
LLM Agents for Time-Series: A Survey
Yilong Chen, Xiao Qin, Chenghao Liu et al. (+3 more)
LLM-based
agents are increasingly being developed for time-series problems, but their design choices vary substantially across task settings. This survey adopts a problem-driven taxonomy that organizes these systems by the time-series problems they address rather than by isolated technical components. We group existing systems into four categories:
forecasting and
reasoning, augmentation and synthesis, anomaly detection and diagnosis, and decision support. Within each category, we examine how task requirements shape
agent architecture, tool use, and
memory design. We further summarize representative
datasets and environments, and compare reported model performance under shared or closely related settings. Overall, this survey offers a task-oriented guide to designing LLM-based
agents for time-series problems and identifies open gaps for future work.
Read the full abstract ▾
All authors Yilong Chen, Xiao Qin, Chenghao Liu, Liang Wu, Noelle I. Samia, Kaize Ding
Artificial Intelligence
Agent Mesh: Reliability Primitives for Non-Idempotent Agent Delegation - Identity Adequacy and Evidence Adequacy
Mazhar Shaikh, Anurag Rajkumar Bombarde and Harshal Pathak
Autonomous
agents increasingly perform bounded software tasks under an orchestrator that retries, resumes, and budgets them. The machinery such orchestrators reach for is the service mesh's: retry, timeout, and error-rate
circuit breaking. We report a failure study of a production agentic software-delivery platform over 147 numbered incidents spanning 81 runs, each with a measured cost and, in most cases, a mutation proof reproducing the failure. All three assumptions those primitives rest on are violated in practice, and we quantify the consequences: a loop of fifty-four consecutive successful tool calls no error-rate breaker could see; a progress signal constant by construction, guaranteeing a false trip on the third repair round and driving one run from six of six components to three; twenty-one events accumulated across six invocations of one delegation, making a correct, idempotent component unwinnable; a misrouted failure that woke five components for a two-component fault, leaving three bystanders regressing working code; and twelve incidents in which the enforcement layer blocked correct work, the most expensive costing 107
agent turns and zero accepted writes. We find one cross-cutting cause and its dual. Identity adequacy: in five separate subsystems an identity that failed to discriminate produced a confident wrong answer, and two of them derived the corrective rule independently. Evidence adequacy: a reliability decision may be taken only on evidence capable of moving, attributable to what it measures, and deterministic under identical conditions. From the findings we derive seven reliability primitives whose enforcement unit is the delegation rather than the message, and specify the controlled evaluation the study motivates but does not constitute.
Read the full abstract ▾
Machine Learning · Cross-listed from hep-ph
Classical and Hybrid Quantum Machine Learning for Trigger-Like Event Selection on CMS Open Data: An Eight-Qubit, PCA-Constrained Benchmark
Tariq Mahmood, Muhammad Awais Rafique, Talab Hussain et al. (+3 more)
Event triggering sits at the heart of high-energy physics, where the rare events of interest must be retained while an overwhelming background is discarded under tight latency and bandwidth budgets. This work compares four classical machine learning models, namely a support vector machine, an artificial neural network, a convolutional network and a long short-term
memory network, with four hybrid
quantum counterparts, on a trigger-like binary classification task built from CMS open data. The label is defined by an invariant-mass window, and the inputs combine reconstructed kinematics with physics-motivated derived variables: the pseudorapidity difference, the wrapped azimuthal difference, the angular separation and the total transverse momentum. The
quantum models run under a fixed resource budget of eight
qubits, a principal-component compression to sixteen features and state-vector simulation. Every model shares the same stratified split, the same preprocessing and a common decision threshold, and performance is reported through accuracy, ROC-AUC, F1-score, precision and recall. The strongest classical model is the artificial neural network, at 93.53 percent accuracy and 0.9819 ROC-AUC, while the strongest
quantum model is the
quantum convolutional network, at 90.89 percent accuracy and 0.9731 ROC-AUC, with the
quantum neural network close behind. The
quantum-kernel and recurrent
quantum approaches trail both, which places the trainable hybrid
embeddings ahead within this budget. The study is meant as a controlled reference point rather than a claim of
quantum advantage.
Read the full abstract ▾
All authors Tariq Mahmood, Muhammad Awais Rafique, Talab Hussain, Juan Pablo Perez Aguilar, Alfredo Raya, Muhammad Ahsan
Hardware Architecture
Beyond Edge Cuts: Activity-Weighted Multicast Hypergraph Mapping for Spiking Neural Networks on Mesh NoCs
Amirreza Khorasanian
Mapping spiking neural networks (SNNs) onto neuromorphic many-core platforms is often formulated with graph partitioning and pairwise placement costs. That abstraction is convenient, but it does not match the physical communication event: one spike from a source
neuron is delivered to a set of postsynaptic destinations, and routes to several destinations can share mesh links. We present M-HySMap, a route-aware, activity-weighted multicast hypergraph mapping framework. Each source
neuron induces a directed hyperedge to its postsynaptic fanout, weighted by profiled activity. The mapper starts from strong activity-aware graph/QAP seeds and then optimizes distinct destination-core fanout, the union of deterministic mesh routes, and link congestion. The central algorithmic observation is locality: moving one
neuron can change only its own source-rooted hyperedge and the hyperedges of its predecessors. This permits exact incremental gain evaluation while caching every unaffected route contribution. We expose this combinatorial structure in detail, derive a conservative placement lower bound, and describe a portfolio of partition and placement neighborhoods that preserves the best incumbent. Across a 115-job evidence suite on Potjans-inspired recurrent SNNs and mesh NoCs from 4 x 4 to 6 x 6, plus a 7 x 7 stress case, M-HySMap reduces routed multicast hops by 10.6-19.6% over Activity+QAP and 19.7-41.1% over Edge+QAP. Incremental updates accelerate refinement by 4.7-12.7x while matching full recomputation to numerical precision.
Read the full abstract ▾
Machine Learning
NeuronFuzz: Safety Neuron Guided Fuzzing for LLM Safety Evaluation
Zhiyuan Xu, Muhammad Firhard Roslan, Joseph Gardiner et al. (+2 more)
Safety evaluation is critical for assessing whether aligned Large
Language Models (LLMs) remain robust against jailbreak attacks. Existing automated testing methods, however, largely rely on response-level feedback: each candidate
prompt typically requires generating a target-model response to evaluate its attack effectiveness. This process is expensive and, more importantly, provides only sparse guidance on strongly aligned models, where most candidates are rejected with the same failure outcome. This paper presents NeuronFuzz, a white-box fuzzing framework that exploits internal safety
neurons as continuous execution feedback for LLM safety evaluation. A SafetyOracle converts safety-
neuron activations into a continuous safety alarm score that serves as feedback for fuzzing and can be obtained during prefill, eliminating response generation from the fuzzing loop. To construct the SafetyOracle, NeuronFuzz uses template-invariant harmful and benign inputs and stability-aware selection to identify a compact set of safety
neurons whose activations capture harmful-intent recognition. Moreover, since the safety alarm score is differentiable, NeuronFuzz uses its gradients to identify safety-sensitive template positions and a masked
language model to generate fluent, context-compatible mutations while preserving original harmful payload and avoiding additional optimization variables. We evaluate NeuronFuzz across 21 text and
multimodal models. Across five white-box source models, it achieves a 76-100% jailbreak discovery rate, outperforming baselines by up to 48 percentage points. Its optimized templates further transfer zero-shot to open-weight and six proprietary target models, achieving average ASR and top-5 ensemble ASR (EASR) of 69.6%/92.6% and 44.1%/60.0%, respectively.
Read the full abstract ▾
All authors Zhiyuan Xu, Muhammad Firhard Roslan, Joseph Gardiner, Sana Belguith, Lichao Wu
Artificial Intelligence · Cross-listed from physics.soc-ph
Prompt Sensitivity of Generative Agents: Evidence from an Epidemic Model
Ross Williams and Niyousha Hosseinichimeh
As generative AI gains traction, researchers are investigating its potential to serve as proxies for humans. From undergoing cognitive psychology experiments to experiencing an epidemic, generative
agents,
agents powered by generative AI models, produce realistic human behavior when prompted. This study explores the sensitivity of these generative
agents' behavior to
prompt modifications and varied persona names of the
agents. To assess this sensitivity, we use a generative
agent epidemic model, wherein each
agent is prompted daily on whether it wants to isolate or commingle with other
agents. We found that using synonymous
prompts results in negligible changes to the model's outcomes. However, minor variations in
prompts, as well as contextual changes, do influence the model's results. Lastly, our data indicates that different persona names assigned to generative
agents, specifically those imbued with personas, do not significantly impact epidemic outcomes.
Read the full abstract ▾
Hardware Architecture
MeshReduce-U: Compiler-Guided Communication Reduction for Irregular Neural Reductions on Mesh NoCs
Amirreza Khorasanian
Many irregular neural workloads induce skewed many-to-one reductions with repeated neighborhoods and nonlocal communication. Conventional NoC mappers optimize placement and routes for a fixed communication graph, even though associative reductions expose legal opportunities to eliminate traffic before routing. We present MeshReduce-U, a compiler-guided communication-reduction and routing framework for mesh-NoC-based spatial accelerators. MeshReduce-U coalesces colocated sources, forms local aggregation islands, blocks channels with compatible fan-in structure, selects capacity-feasible sinks, and routes the remaining fixed-width carriers using fused usage-aware costs. A deterministic route-replay model reports schedule-derived communication latency, total link usage (TLU), and fused link usage (FusedTLU) separately. Across a 20-workload lowerable neural-network zoo, MeshReduce-U reduces mean latency, TLU, and FusedTLU by 40.3%, 56.0%, and 48.7%, respectively, relative to an ABC-style source-aggregation baseline, improving all three metrics on every workload. Across 40 synthetic irregular reductions, it reduces mean latency and TLU by 12.3% and 19.7%. A new 30-instance pass-by-pass study further shows that the structural rewrites reduce the global carrier count by 60.9% and replay latency by 63.0%. These results show that rewriting reducible neural communication before routing can be more effective than searching harder over an unreduced traffic graph.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
TRACE: Retrospective Streaming Generation of Physical Fields under Sparse Structured Sensing
Xinyu Zhang, Lihao Chen, Panqi Chen et al. (+4 more)
Reconstructing continuous physical fields from sparse measurements is central to scientific monitoring, inverse modeling, and digital-twin construction. Generative reconstruction has recently emerged as a promising paradigm for this task by learning data-driven physical priors that complete plausible full fields from limited observations. However, existing methods largely assume fixed, batch conditioning, whereas real sensing systems often produce structured streams: probes scan local regions, instruments observe moving fields of view, and communication constraints may leave entire frames missing. We propose TRACE, a retrospective streaming generative reconstruction framework for physical fields under structured sensing. TRACE performs approximate
Bayesian inference in a learned continuous-coordinate latent space, converting sparse off-grid measurements into generative latent evidence, fusing it with a state-space temporal prior through Kalman-style filtering, and refining under-observed past frames via retrospective smoothing. Experiments on active matter, ocean sound-speed fields, and
supernova simulations show that TRACE matches or surpasses frame-wise generative reconstructors, offline spatiotemporal methods, and streaming data-assimilation baselines in reconstruction quality under temporally sparse and spatially localized sensing protocols.
Read the full abstract ▾
All authors Xinyu Zhang, Lihao Chen, Panqi Chen, Lei Cheng, Ting Zhang, Jianlong Li, Shikai Fang
Artificial Intelligence
Same Model, Different Harness: Different Coding-Agent Results
Sydney Lewis
A coding
agent combines a model with a harness, which decides what the model sees, which tools it can use, and how the work continues. We ask whether changing the harness changes the result when the model and task stay fixed. We compare two configurations of the same harness on three coding
benchmarks. The control supplies the full conversation in time order, while the treatment keeps the same record but mechanically shortens older tool results as the context fills and responds to repeated or stalled work. Under tight context, the treatment raises mean per-task fail-to-pass fraction (F2PF) in all three pressure comparisons and increases complete solutions on SWE-bench Verified and SWE-bench Pro. The tight-window Verified comparison uses 169 tasks, a 20,480-
token window, and a fixed 480-second attempt endpoint; on this cohort, treatment raises mean per-task F2PF from 28 percent to 49 percent and complete solutions from 43 to 72. Without model-specific retuning, the same frozen treatment also raises both endpoints on the same cohort for three additional models with different designs. In the wide-window Qwen3.6 comparisons, observed arm outcomes are close on Verified and Pro, while FeatureBench retains a higher mean per-task F2PF under treatment. On the wide-window Verified cohort, treatment also serves fewer
prompt tokens per turn. Because changing the harness changed what unchanged model weights could accomplish, coding-
agent evaluations should treat the model and harness together as the tested solver.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.plasm-ph
Real-time virtual circuits for plasma shape control via neural network emulators: integration and testing in the MAST-U PCS
Matthew J. Marshall, Edward Jones, Graham J. McArdle et al. (+10 more)
The deployment of advanced, AI-enabled control
algorithms in tokamak experiments requires robust integration with existing
plasma control system (PCS) architectures and extensive pre-experimental validation. In this contribution, we describe the integration and testing of neural-network-emulated virtual
circuits for
plasma shape control within the MAST Upgrade (MAST-U) PCS environment. The neural network models predict the
plasma shape using the
plasma current, poloidal field coil currents, and
plasma profile parameters. In this paper, we explain how they are deployed via a real-time C++ inference server that interfaces with the PCS, returning the shape prediction and its Jacobian, and how, from the latter, virtual
circuit matrices and updated coil current requests are computed for real-time actuation. Emphasis is placed on the validation workflow and best practices adopted to ensure confidence in the proposed control framework prior to experimental deployment. This work demonstrates practical AI-based shape control components for fusion control systems, with direct relevance for upcoming MAST-U experiments and future devices.
Read the full abstract ▾
All authors Matthew J. Marshall, Edward Jones, Graham J. McArdle, Alasdair Ross, Kamran Pentland, Nicola C. Amorisco, Charles Vincent, Martin Kochan, Colin Hogben, Graham Jones, Adam Stephen, George K. Holt, Adriano Agnello
Software Engineering
Challenges and Contributions in Quality of AI-Based Software: A Systematic Mapping Study
Maryum Hamdani, Mateen Ahmed Abbasi, Marko Jäntti et al. (+1 more)
Artificial Intelligence (AI) is increasingly embedded in modern software systems, raising important questions about how its quality should be defined, assessed, and assured. This paper presents a Systematic Mapping Study (SMS) on the quality of AI-based software. The study synthesizes primary studies published between January 2020 and January 2026 and selected from five electronic data sources. A total of 33 primary studies were included after automated search, screening, and snowballing. The results identify six recurring challenge categories, with the most prominent being limitations in existing quality assessment models, followed by issues in non-functional requirement management, quality-aware development, and quality assurance. The findings suggest a call for collaboration of researchers and industrial practitioners with standardization organizations, that could possibly devise comprehensive quality assessments and their measurement methods.
Read the full abstract ▾
All authors Maryum Hamdani, Mateen Ahmed Abbasi, Marko Jäntti, Markku Tukiainen
Computer Vision
Fuxiang Huang, Chenxu Zhang, Liang Han et al. (+1 more)
Surgical video data provides the primary training resource for models of intraoperative perception,
surgical workflow understanding, and robotic decision-making. However, clinical data acquisition remains constrained by
privacy, cost, and class imbalance.
Surgical video generation has emerged as a transformative approach to addressing data scarcity and as a foundation for
surgical simulation, training, and robotic policy learning. The field has developed rapidly without a clear conceptual framework. This survey organizes the 2024-2026 literature into three categories: unconditional generation, conditional generation, and world modeling generation, revealing a fundamental shift in how the task is defined from synthesizing visually plausible frames to modeling the causal dynamics of
surgical scenes. We examine the persistent gap between pixel-level fidelity and clinical plausibility, and identify generalization, physical realism, controllability, and interpretability as bottlenecks. We further summarize experimental results of representative methods on public
datasets to provide a quantitative reference for the field. This survey provides a structured overview of the current state and open challenges, offering a reference for researchers working at the intersection of intelligent perception, multi-modal fusion, generative AI, and
surgical data science.
Read the full abstract ▾
All authors Fuxiang Huang, Chenxu Zhang, Liang Han, Lei Zhang
Sound
Attention-Guided Reliability Scaling for Contrastive Decoding in Robust Audio-Visual Speech Recognition
YoungChae Kim, Da-Hee Yang and Joon-Hyuk Chang
Large
language model (LLM)-based audio-visual
speech recognition (AVSR) systems are robust under noise. Contrastive decoding (CD), originally introduced to stabilize LLM generation by contrasting a weaker model against a stronger one at inference time, adjusts predictions without additional training. In this work, we apply CD to AVSR by contrasting audio-only conditioning with full audio-visual conditioning within the same underlying model. However, using a fixed contrastive strength introduces a trade-off across noise levels: stronger intervention helps under severe noise but may over-correct reliable predictions in clean conditions. We propose reliability-aware scaling of CD for AVSR. Instead of using a fixed strength, we adaptively modulate the contrastive influence at each
token based on reliability signals derived from
attention dynamics and inter-model predictive divergence. Experiments on LRS3 show consistent improvements across clean and low-SNR conditions.
Read the full abstract ▾
Software Engineering
Characterizing the Landscape of Open-Source Satellite Software
Jinfeng Wen, Qi Liang, Yuehan Sun et al. (+4 more)
Satellites have become fundamental components of modern technological systems, supporting critical infrastructure in communication,
navigation, Earth observation, and scientific research. As space exploration advances and demand for satellite-enabled services grows, reliance on complex, heterogeneous satellite software continues to increase. A systematic understanding of the satellite software landscape is therefore increasingly important, yet existing studies still lack a comprehensive empirical examination. To address this gap, we present the first characterization study of open-source satellite software, examining its ecosystem and development practices. We mine and analyze 22,286 satellite-related GitHub projects through three research questions on popularity trends (RQ1), software goals (RQ2), and development practices (RQ3). First, we characterize the temporal
evolution of projects and active developers, revealing increasing popularity. Second, through manual inspection of 646 projects, we construct a taxonomy of 43 software-goal categories spanning conceptual design,
datasets, system implementation, simulation, testing, and tools. Third, we conduct an in-depth analysis of projects with source code, revealing a highly heterogeneous and task-specialized ecosystem with 66 programming
languages and diverse implementation strategies. Finally, we summarize key findings and derive actionable implications for satellite developers and researchers.
Read the full abstract ▾
All authors Jinfeng Wen, Qi Liang, Yuehan Sun, Federica Sarro, Ao Zhou, Xuanzhe Liu, Shangguang Wang
Software Engineering
Fairness Invariants: A Relational Approach to Explaining and Mitigating Fairness Bugs
Ranit Debnath Akash, Ashish Kumar, Gang Tan et al. (+1 more)
Data-driven software systems are increasingly deployed in high-stakes socio-economic domains, from criminal justice to financial lending. However, these systems often exhibit individual discrimination—unjustified disparities in which a program yields different outcomes for similar individuals who differ only in their protected attributes (e.g., race, gender, age). While existing research has focused on detecting and quantifying these bugs, there remains a critical lack of principled mechanisms to explain and localize individual
fairness bugs. Current explanation techniques are largely designed for single-input decisions rather than the relational nature of discrimination, which inherently involves a comparison between an original and a counterfactual pair. We present REMI, a framework for the automated localization, explanation, and mitigation of individual discrimination. Inspired by loop-invariant synthesis in formal methods, we treat counterfactual
fairness as a relational invariant discovery problem. We introduce a bidirectional relational explanation framework that learns over paired examples $(x, x')$ to identify regions of the input space where
fairness is violated. Unlike traditional one-way implication pairs used in invariant inference, our approach enforces bidirectional constraints: requiring identical outcomes for both original and counterfactual samples. REMI utilizes three data-
alignment techniques to infer interpretable rule-based models that act as "
fairness invariants." These rules serve as guardrails to selectively block or relabel unfair predictions without requiring model retraining. Our evaluation on symbolic and neural network programs demonstrates that REMI localizes ground-truth
fairness bugs in over 83% of cases, significantly outperforming state-of-the-art baselines and reducing discriminatory decisions in black-box models by up to 70%.
Read the full abstract ▾
All authors Ranit Debnath Akash, Ashish Kumar, Gang Tan, Saeid Tizpaz-Niari
Hardware Architecture
Ankhdjet: An Open-Source Compiler for Mask-Programmed Ternary Compute-in-ROM on an Open PDK
Mohnish Pai
Large-
language-model inference is dominated by weight movement: every generated
token re-reads every weight. Ternary quantization (BitNet b1.58) shrinks each weight to 1.58 bits with reported parity at the 2B-parameter scale, small enough that hardwiring the weights into a read-only mask becomes plausible, and a commercial chip (Taalas HC1) has validated hardwired weights on an advanced node behind closed tooling. This paper asks whether model-specific silicon can be made reproducible with entirely open infrastructure. We present Ankhdjet, an open-source compiler that lowers a HuggingFace ternary checkpoint (BitNet b1.58 and its kin) to a via-mask program of a fixed compute-in-ROM macro on the open SKY130 PDK, verified end to end with open tools. We defend two claims: (1) the first open-source weights-to-mask compute-in-ROM compiler on a fabricable open PDK, taken through full open-toolchain signoff (KLayout DRC zero, netgen LVS zero, clean timing) twice with two different weight
matrices through an identical flow in which only the mask differs; and (2) the first compute-in-ROM macro submitted for fabrication on an open PDK (TinyTapeout/ChipFoundry SKY130 shuttle ttsky26c, silicon expected 2027). The compiler's read is fully digital (full-swing bitline sampling into standard
cells, so the emission ports to any node by synthesis); an analog comparator readout of the same machine is retained as a measured variant, replaced by the digital readout at one eighth the area. We additionally report the per-level
adversarial verification methodology that caught three DRC-invisible silent shorts, a negative result on storing two ternary weights per transistor at 130 nm, and which analog sense techniques survive at which nodes. Read energy is from extracted-parasitic simulation (0.98-1.73 pJ per sensed weight); no energy measurements are claimed, and the fabricated vehicles are demonstrators.
Read the full abstract ▾
Computer Vision
VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
Junxiang Xu, Ruisi Wang, Fanyi Pu et al. (+49 more)
Native visual
reasoning treats visual generation as the medium of
reasoning itself: visual states (i.e. images and
videos) are not merely inputs to be understood or outputs to be rendered, but first-class substrates for problem solving beyond
language. Yet progress remains bottlenecked by the lack of scalable training tasks, reliable feedback, and controlled comparisons across generative substrates. In this work, we introduce VBVR-Pro, a closed-loop testbed that makes native visual
reasoning through generation trainable, verifiable, optimizable, and experimentally controllable. 1) Task scaling. VBVR-Pro turns visual
reasoning into a controlled task space of 300 procedurally generated tasks. Models trained on VBVR-Pro show strong transfer beyond the proposed suite across seven external visual
reasoning benchmarks such as RISE-
Video, MME-CoF-Pro, and BabyVision. 2) Verifiable rewards. VBVR-Pro provides verifiable reward scorers for task-grounded evaluation. Through a systematic study of leading MLLMs as judges, we identify recurring failure modes of the prevalent VLM-as-a-judge paradigm. In contrast, the proposed scorers are grounded in deterministic, task-specific rules, achieve fine-grained
alignment with human judgments. Importantly, they serve as reliable reward signals for large-scale multi-task reinforcement learning and demonstrate stronger post-RL performance across visual
reasoning tasks. 3) Mechanism study. VBVR-Pro enables controlled modality studies across more than 30 image,
video, and interleaved generators. Our analysis shows that
video generation remains strongest for tasks requiring persistent spatiotemporal state tracking, while interleaved generation provides a compute-efficient alternative. Critically, ablations and probing suggest the presence of
vision-native trajectories that are crucial to visual
reasoning. We release all data, models, scorers, and code.
Read the full abstract ▾
All authors Junxiang Xu, Ruisi Wang, Fanyi Pu, Maijunxian Wang, Ran Ji, Tongxi Zhou, Chenyang Gu, Jing Zuo, Hongcan Xiao, Yimeng Geng, Wanqi Yin, Wei Chen, Oscar Qian, Zhengan Yan, Ziqi Huang, Haiwen Diao, Liang Pan, Bo Li, Xiangyu Fan, Dezhi Luo, Fengyuan Yu, Zehong Zhao, Qingying Gao, Tinghui Zhu, Yilan Zhang, Jingqi Tong, Pinyuan Feng, Zhengze Jiang, Letian Wang, Ziyu Guo, Renrui Zhang, Jieneng Chen, Sonia Joseph, Constantin Venhoff, Saman Motamed, Mengyue Yang, Chandra Sripada, Alan Yuille, Philip Torr, Lvmin Zhang, Vikash Kumar, Daniel Khashabi, Nikolaus Kriegeskorte, Raphaël Millière, Vincent C. Müller, Anyi Rao, Quan Wang, Ziwei Liu, Dahua Lin, Lei Yang, Hokin Deng, Zhongang Cai
Robotics
Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization
Jiaming Zhou, Qihang Zhang, Gangwei Xu et al. (+9 more)
Zero-shot cross-task generalization, where a policy must execute manipulation tasks never seen during training, remains a central challenge in
robot learning. In large
language models, a novel task can be performed simply by specifying it in the context, without any parameter update. This form of in-context learning (ICL) turns generalization into a problem of task specification. To achieve cross-task generalization, we bring this paradigm to robotic manipulation, and argue that the natural task specification for manipulation is a human
video: unlike
language, it provides rich visual cues about the intended task
evolution. We present Zero-WAM, a causal
video-action model that executes unseen tasks by following in-context human
video guidance. To address the scarcity of task-rich paired human-
robot data, we propose an automatic pipeline that converts task-sampled
robot trajectories into semantically matched human
videos, yielding HumanGen, a
dataset of 74.2K human-
robot ICL pairs across 8.6K tasks. For model training, we further introduce an in-context future chunk prediction (IFP) objective that suppresses shortcuts learned from seen tasks and forces the policy to draw task information from the
video prompt. On seven unseen tasks in RoboTwin 2.0 simulation, Zero-WAM achieves a 47.0% average success rate, an absolute improvement of 29.5 percentage points over the strongest
video-action baseline. In real-world evaluations, it follows human
video guidance to generalize to unseen task configurations involving multi-object scenes, long-horizon manipulation, and fine-grained insertion.
Read the full abstract ▾
All authors Jiaming Zhou, Qihang Zhang, Gangwei Xu, Cunxin Fan, Yujie Zhao, Ruilin Wang, Yiming Luo, Shuai Yang, Xing Zhu, Yujun Shen, Junwei Liang, Yinghao Xu
Computer Vision
RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing
Bojia Zi, Xiaoyan Yang, Yu Zhou et al. (+7 more)
Recent advances in
video editing have been largely driven by large-scale instruction-based
datasets. However, existing
datasets still suffer from two critical limitations. First, target
videos are commonly produced by automatic editing models, which may introduce visible artifacts and unreliable supervision signals. Second, most public
datasets rely primarily on textual instructions, while lacking visual references that are crucial for precise, identity-preserving, and controllable editing. To address these limitations, we introduce RefVideo-6M, a large-scale reference-guided editing
dataset containing 5 million
video editing samples and 1 million image editing samples. To ensure reliable supervision, our
dataset uses a construction pipeline that treats artifact-free real
videos as editing targets and generates quality-filtered input conditions with multiple editing experts. In addition, it provides approximately 6 million visual references, covering diverse reference types and editing scenarios, thereby enabling models to learn fine-grained visual correspondence beyond text-only instructions. Based on RefVideo-6M, we further train a reference-guided
video editing model, Ref-MoT, to evaluate the effectiveness and scalability of the proposed
dataset. Extensive experiments demonstrate that RefVideo-6M provides substantially more reliable supervision than existing
datasets and enables the training of powerful editing models with improved visual quality, controllability, and reference consistency. The open-source
dataset is available at https://huggingface.co/
datasets/RefVideo6M/RefVideo6M.
Read the full abstract ▾
All authors Bojia Zi, Xiaoyan Yang, Yu Zhou, Ruijie Sun, Lihan Zhang, Bin Liang, Kam-Fai Wong, Haibin Huang, Chi Zhang, Xuelong Li
Hardware Architecture
Integrated Hardware Annealing based on Langevin Dynamics for Ising Machines
Yongchao Liu, Lianlong Sun, Michael Huang et al. (+1 more)
Ising machines are non-von Neumann machines designed to solve combinatorial optimization problems (COP) by searching for the ground state, or the lowest energy configuration, within the Ising model. However, Ising machines often face the challenges of getting trapped in local minima due to the complex energy landscapes. Hardware annealing
algorithms help mitigate this issue by using a probabilistic approach to steer the system toward the ground state. In this paper, we present a hardware annealing
algorithm for Ising machines based on Langevin dynamics, a
stochastic perturbation by random noise. Theoretical analysis, system-level design, and detailed
circuit design are carried out. We evaluate the performance of the
algorithm through chip-level simulation using a standard 65-nm CMOS technology to demonstrate the
algorithm's efficacy. The results show that the proposed hardware annealing
algorithm effectively guides the system to reach the ground state with a probability of 86.5%, significantly improving the solution quality by 97.5%. Further, we compare the
algorithm with state-of-the-art hardware annealing methods through behavioral-level simulations, highlighting its improved solution quality alongside a 50% reduction in time-to-solution.
Read the full abstract ▾
All authors Yongchao Liu, Lianlong Sun, Michael Huang, Hui Wu
Computer Vision
A Visual Dependence-Aware Framework for Multimodal Unsupervised Continual Post-Training
Kaichen Li, Zhilin Zhu, Jianhao Huang et al. (+7 more)
In this paper, we explore a novel task of
Multimodal Unsupervised Continual Post-Training (MU-CPT), enabling deployed MLLMs to continually evolve from streaming unlabeled data. Existing unsupervised post-training methods for MLLMs typically optimize target
tokens uniformly, overlooking their heterogeneous visual dependence (VD). However, we reveal that
token-level VD is crucial for MU-CPT. Specifically, its structural distortion serves as an indicator of cross-modal catastrophic forgetting, and its inherent heterogeneity acts as a compass to guide new-task learning. Leveraging this property, we propose a Visual Dependence-Aware (VDA) framework with two main components. First, Visually Constrained Optimal Transport (VC-OT) formulates the VD structural distortion of old-task VD during new-task learning as an optimal transport problem to mitigate cross-modal forgetting. By designing a region-aware ground cost and a dependence-stratified transport penalty, it prevents global shifts in visual focus while strictly prohibiting visual reliance from degenerating into
language bias. Second, Visually Modulated Adaptation (VMA) exploits VD heterogeneity to emphasize visually grounded new-task learning, promoting new-task plasticity. Together, our method simultaneously maintains old-task stability and new-task plasticity during challenging MU-CPT. Extensive experiments under our MU-CPT setting validate the effectiveness of VDA.
Read the full abstract ▾
All authors Kaichen Li, Zhilin Zhu, Jianhao Huang, Zhengqin Lai, Baochen Xiong, Zibo Shao, Yaguang Song, Linhui Xiao, Xiaoshan Yang, Changsheng Xu
Computer Vision
MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching
Hao Yin, Paritosh Parmar, Lijun Gu et al. (+6 more)
Existing action quality assessment (AQA)
datasets and methods rely primarily on visual inputs such as RGB and pose, overlooking physiological dynamics such as muscle mechanics and often modeling actions as monolithic patterns. These limitations hinder fine-grained, biomechanically grounded feedback. We introduce MyoMechanix, a
multimodal ecosystem for weight-loaded actions that aligns motion with muscle activity. Expert-annotated, it contains 7,500+ samples of 20 actions from 38 subjects, with synchronized multiview RGB
video, 3D pose, sEMG, and additional physiological signals, forming the largest
multimodal AQA
benchmark to date. We further construct the Fitness Knowledge Graph (FKG), which organizes expert annotations into structured relationships among actions, phases, key steps, errors, and corrective feedback, enabling compositional scoring and interpretable assessment. Building on these representations, we develop CUBIST (Compositional Ontological
Reasoning Engine), which performs decomposition-analysis-recomposition for fine-grained error attribution and feedback generation. We also establish MyoMechanix-AQA, MyoMechanix-VideoQA, and a novel MyoMechanix-
Video2EMG task. Experiments show that
multimodal sensing and structured representations improve performance, interpretability, and error attribution, with CUBIST achieving state-of-the-art results; VideoQA enhances
language-grounded action understanding; and
Video2EMG suggests
video-based alternatives to costly EMG sensing. MyoMechanix advances skilled activity understanding toward biomechanically grounded,
multimodal, and compositional
reasoning for Physical AI applications in fitness, rehabilitation, healthcare, and machine learning. Project page: https://haoyin116.github.io/MyoMechanix/
Read the full abstract ▾
All authors Hao Yin, Paritosh Parmar, Lijun Gu, Lin Xu, Tianxiao Guo, Xiujin Liu, Tianyou Zheng, Yang Zhang, Weiwei Fu
Machine Learning
Agentic Autoresearch for Cell-Edge Power Control: Radically Redefining the Researcher's Role
Ahmad Khan, Akram Bin Sediq, Sara Azadegi Naeini et al. (+1 more)
Designing machine learning
algorithms for wireless resource management is labour-intensive: the architecture, the loss function and the training recipe are all specified by hand. We demonstrate that this design layer can be surrendered to an autonomous
agent in its entirety. We adopt the autoresearch protocol, in which an AI coding
agent edits a training script, runs a fixed-budget experiment, and retains or discards the change according to a single immutable metric. We grant the
agent authority over the architecture family, the input representation, the output parameterization, the loss function and the task-sampling law, and set it a target chosen for its difficulty: sum-least-percentile-rate power control across a multicell network. The formulation targets
cell-edge throughput and is non-convex, non-smooth and strongly NP-hard away from its max-min vertex. Safeguards render the results trustworthy: a hash-pinned evaluator, an enforced inference contract and a pre-registered falsifier per experiment. In eighty-one unattended experiments over twenty-six hours, the
agent reached $99.5\%$ of a converged minorization-maximization reference in one fixed-cost inference pass, at roughly $600\times$ lower inference cost, closing $94\%$ of the gap from its first working architecture, with one parameter set serving every network size and percentile target. It recovered provable structure rather than tuned constants: the output parameterization it discovered reproduces the exact max-min-optimal allocation at the minimum percentile, for every value of the trained weights.
Read the full abstract ▾
All authors Ahmad Khan, Akram Bin Sediq, Sara Azadegi Naeini, Raviraj S. Adve
Information Retrieval
PlanSightRAG: A Visual-First Multimodal RAG for Automating Question Answering and Compliance Checking for Civil Standard Plans
Nabaraj Subedi, Shuvo Dip Datta, Ahmed Abdelaty et al. (+1 more)
Civil infrastructure compliance checking has long relied on engineers manually reading legacy 2D plans; however, OCR-based automation strips away the geometry and layout essential for interpreting these plans. We present a Visual-First
Multimodal Retrieval-Augmented Generation (RAG) framework called PlanSightRAG. It indexes and reasons directly over plan imagery, integrates a ColNomic-3B multi-vector
retrieval, an agentic Planner-Retriever-Auditor-Synthesizer, and MaxSim heatmaps as an evidence trail. We introduce a 4,056-pair
benchmark from five state Departments of Transportation (DOT) standard plans (1,898 pages). PlanSightRAG achieves 91.47% Recall@5 on zero-shot
retrieval, while on a held-out Michigan DOT corpus, it achieves 91.40%. On synthetic, parametrically-generated compliance drawings, our Qwen2.5-VL-72B pipeline reaches 100% verdict accuracy only when supplied a pre-resolved rule threshold, a controlled ceiling that a non-VLM OCR baseline already reaches at 76.4%. Finally, we demonstrate autonomous visual rule-grounding by extracting numeric limits directly from a specification corpus without any human-supplied rules.
Read the full abstract ▾
All authors Nabaraj Subedi, Shuvo Dip Datta, Ahmed Abdelaty, Shivanand Venkanna Sheshappanavar
Artificial Intelligence · Cross-listed from astro-ph.HE
Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders
Raphaël Bonnet-Guerrini, Johann Ioannou-Nikolaides, Inar Timiryasov et al. (+1 more)
We present a first application of sparse-autoencoder-based mechanistic interpretability to particle physics. Studying a
neutrino foundation model pretrained on IceCube data and fine-tuned for direction reconstruction, we identify a validated atlas of physical concepts in the model representation, using a strict validation protocol consisting of held-out tests, matched nuisance controls, and replication across independent dictionary trainings. Causal interventions show that the direction head barely draws on this atlas. Motivated by this underused information, we train an uncertainty head on the same event-level representation to predict the model's angular reconstruction error. Unlike the direction head, it depends causally on quality and brightness features from the atlas. These results suggest that mechanistic interpretability can reveal learned latent physics encoded within a model's internal representation and help design downstream tasks that exploit it.
Read the full abstract ▾
All authors Raphaël Bonnet-Guerrini, Johann Ioannou-Nikolaides, Inar Timiryasov, Vincenzo Piuri
Human-Computer Interaction
From Producing to Validating: How AI Is Deskilling Freelancers
Nakul Rajpal
Generative AI is promoted as a way to enhance knowledge work, yet its benefits and drawbacks fall unevenly across the workforce. Freelance and gig workers, who commonly lack the upskilling pathways available to traditional employees, face heightened risks to both skill development and job security as AI adoption advances. We review empirical evidence on AI's impact on knowledge-worker workflows and upskilling, then predict the primary and downstream effects of AI adoption among clients and workers in the freelance economy. We anchor this in two cases of the same shift, machine-translation post-editing and software development. We argue that freelancers are the leading edge of a change that also reaches salaried HCI practitioners, and we close with questions for the platforms and clients that mediate this work, and for HCI researchers.
Read the full abstract ▾
Artificial Intelligence
Planetary Prediction Engine: Autonomous Geospatial Prediction via Intelligent Data Selection and Foundation Model Embeddings
Evelyn Ma, Rama Kumar Pasumarthi, Kishwar Shafin et al. (+26 more)
Addressing critical global challenges, from food security and disaster risk to disease outbreaks and socio-economic vulnerability, demands high-fidelity geospatial modeling. However, building predictive planetary models remains bottlenecked by a fragmented data ecosystem, requiring manual data
retrieval,
multimodal data curation and fusion along with iterative model selection. We present the Planetary Prediction Engine (PPE), an autonomous AI system that executes this end-to-end workflow directly from natural-
language queries. PPE synthesizes
multimodal datasets on the fly, retrieving spatiotemporally relevant
covariates across open-web and Earth observation platforms (Data Commons, Google Earth Engine) and fusing them with geospatial foundation model
embeddings (PDFM, AlphaEarth). Simultaneously, it searches over task-tailored model architecture families with automated overfitting guards. Across diverse tasks, geographies, and scientific domains, PPE consistently outperforms state-of-the-art or manually tuned expert baselines. For US spatial regression, PPE improves mean $R^2$ across 21 CDC health indicators (76.8% vs. 60.0%), FEMA national risk indices (64.9% vs. 60.0%), and the Social Vulnerability Index (66.2% vs. 58.6%). For spatial downscaling in data-scarce settings, PPE integrates localized proxies to double baseline accuracy in Nigerian food security indicators ($R^2$ of 66.1% vs. 31.5%). For epidemiological nowcasting of the 2026 DRC Bundibugyo Ebola outbreak, PPE achieves a Recall@10 of 83.3% (identifying 15 of 18 newly invaded health zones across five weekly forecasts), a +10.3 percentage-point improvement over the public state-of-the-art modeling (~73%). By combining autonomous
multimodal planetary data discovery with targeted model optimization, PPE lowers the technical barrier to planetary-scale analytics, enabling rapid, customized, expert-level deployment.
Read the full abstract ▾
All authors Evelyn Ma, Rama Kumar Pasumarthi, Kishwar Shafin, Mandar Sharma, Mimi Sun, Hamed Sadeghi, Dav M. Ebengo, Mbulayi Onesime, Ciara Judge, Rouslan Solomakhin, John Wamburu, William Ogallo, Aisha Walcott-Bryant, Sanxing Chen, Arbaaz Muslim, Yael Mayer, Ronald Ho, Roy Lee, Ruth Alcantara, Abdoulaye Diack, Monica Bharel, Lambert Rosique, Jeremy Amez-Droz, Christopher Haire, James Manyika, Yossi Matias, Niv Efron, Gautam Prasad, Shravya Shetty
Machine Learning
TraceML: What Auto-Research Agents Miss in Long-Horizon ML Development
Jiarui Yan, Weiwei Sun, Sijie Li et al. (+2 more)
Auto-research
agents now run machine-learning development unattended for hours, revising data pipelines, models, and validation from their own feedback, yet on most competitions they still finish below strong human competitors. Outcome-based
benchmarks record this gap but not its cause, because they grade the final submission and discard the development process behind it. We introduce TraceML, which pairs human and
agent work on the same competitions under one version-level schema: 4,465 human Kaggle trajectories across 134 competitions, seven of which are also worked by two
agent scaffolds, giving 430 paired human and 207
agent trajectories. Every code version carries its score, its timestamp, and labels for the action taken, its intent, the edit size, and the score effect. Read this way, the gap becomes concrete. Experts alternate data work, validation, model changes, and ensembling, and return to approaches they had set aside. Each
agent scaffold instead collapses into a narrow loop: Codex spends its steps re-weighting ensembles and tuning submissions, MLEvolve mutates its model in place, and neither pivots at the human rate nor reopens abandoned work. A short planning
prompt distilled from human practice moves the behaviors it names toward the human profile and lifts scores, but the effort profile stays
agent-shaped: instruction closes only the part of the gap that reduces to instructions. We release the corpus, the schema, and the labeling models, together with an open-source toolkit that turns a run from any command-line
agent into a TraceML trajectory and reads it against the human cohorts.
Read the full abstract ▾
All authors Jiarui Yan, Weiwei Sun, Sijie Li, Wenhan Li, Yiming Yang
Machine Learning
ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on
Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer et al. (+7 more)
Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Before deployment, models should therefore be audited for reliance on concepts such as acquisition artifacts and patient demographics. Current methods, including linear probes and concept activation vectors, measure concept reliance by asking whether each concept is decodable from a layer. We show that this framing is misleading: decodability reflects not only concept reliance but also correlations among concepts and the outcome in the
auditing dataset. We reframe shortcut
auditing as a decomposition of the variance of intermediary representations with Independent Canonical cONcepts (ICON). ICON quantifies the share of a layer's variance explained by each concept conditional on all other concepts and the outcome, and also reports the variance left unexplained. Because ICON explanations are comparable across layers, their trajectories distinguish concepts that drive predictions from those that are merely encoded in the representations. In simulated data with ground truth, ICON recovers concept importance more accurately than seven baselines. In skin-cancer and neuroimaging applications, it separates learned shortcuts from merely correlated concepts, which we confirm with independent validation checks.
Read the full abstract ▾
All authors Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer, Marc-Andre Schulz, Nys Tjade Siegel, Maximilian Dreyer, Frederik Pahde, Wojciech Samek, Sonja Greven, Kerstin Ritter
Artificial Intelligence
SwarmWorld: Stigmergic technological evolution in societies of language-model agents
Subhadeep Pal, Fiona Y. Wang and Markus J. Buehler
Collective intelligence can emerge when individuals coordinate through a shared environment, allowing local actions to accumulate into durable social organization.
Language-model
agents offer a new substrate for this process, yet most multi-
agent systems rely on direct conversation, predefined roles, or centralized workflows. It remains unclear whether decentralized
agents can build functional technologies and outperform independent search. Here, initially homogeneous LLM
agents in SwarmWorld self-organize without assigned roles or recipes into evolving technological societies.
Agents explore a spatial environment, process resources, test materials, construct persistent artifacts, and write executable controllers evaluated by a deterministic simulator under unseen disturbances after the
agents are removed. SwarmWorld splits cognition from consequence:
agents propose architectures and controllers within fixed action and material schemas, while the simulated world determines function. Shared societies develop broader, more resilient technological portfolios than a strong best-of-N isolated-search baseline, although isolated search remains competitive for the strongest artifact.
Agents differentiate into exploration, construction, maintenance, and coordination behaviors, transitioning as the world matures. Technologies accumulate through collaborative construction, executable inheritance, and persistent
agent-artifact networks, with most reuse beginning through physical observation rather than communication. Explicit cultural mechanisms amplify collaboration and organization, but functional benefits depend on outcome and timescale. Physical stigmergy alone supports capable societies, while interaction drives persistent technological ecologies rather than universally superior individual inventions.
Read the full abstract ▾