Artificial Intelligence · Cross-listed from cs.CV
MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching
Hao Yin, Paritosh Parmar, Lijun Gu et al. (+6 more)
Existing action quality assessment (AQA)
datasets and methods rely primarily on visual inputs such as RGB and pose, overlooking physiological dynamics such as muscle mechanics and often modeling actions as monolithic patterns. These limitations hinder fine-grained, biomechanically grounded feedback. We introduce MyoMechanix, a
multimodal ecosystem for weight-loaded actions that aligns motion with muscle activity. Expert-annotated, it contains 7,500+ samples of 20 actions from 38 subjects, with synchronized multiview RGB
video, 3D pose, sEMG, and additional physiological signals, forming the largest
multimodal AQA
benchmark to date. We further construct the Fitness Knowledge Graph (FKG), which organizes expert annotations into structured relationships among actions, phases, key steps, errors, and corrective feedback, enabling compositional scoring and interpretable assessment. Building on these representations, we develop CUBIST (Compositional Ontological
Reasoning Engine), which performs decomposition-analysis-recomposition for fine-grained error attribution and feedback generation. We also establish MyoMechanix-AQA, MyoMechanix-VideoQA, and a novel MyoMechanix-
Video2EMG task. Experiments show that
multimodal sensing and structured representations improve performance, interpretability, and error attribution, with CUBIST achieving state-of-the-art results; VideoQA enhances
language-grounded action understanding; and
Video2EMG suggests
video-based alternatives to costly EMG sensing. MyoMechanix advances skilled activity understanding toward biomechanically grounded,
multimodal, and compositional
reasoning for Physical AI applications in fitness, rehabilitation, healthcare, and machine learning. Project page: https://haoyin116.github.io/MyoMechanix/
Read the full abstract ▾
All authors Hao Yin, Paritosh Parmar, Lijun Gu, Lin Xu, Tianxiao Guo, Xiujin Liu, Tianyou Zheng, Yang Zhang, Weiwei Fu
Artificial Intelligence · Cross-listed from astro-ph.HE
Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders
Raphaël Bonnet-Guerrini, Johann Ioannou-Nikolaides, Inar Timiryasov et al. (+1 more)
We present a first application of sparse-autoencoder-based mechanistic interpretability to particle physics. Studying a
neutrino foundation model pretrained on IceCube data and fine-tuned for direction reconstruction, we identify a validated atlas of physical concepts in the model representation, using a strict validation protocol consisting of held-out tests, matched nuisance controls, and replication across independent dictionary trainings. Causal interventions show that the direction head barely draws on this atlas. Motivated by this underused information, we train an uncertainty head on the same event-level representation to predict the model's angular reconstruction error. Unlike the direction head, it depends causally on quality and brightness features from the atlas. At $20\%$ selection efficiency, this interpretable
estimator improves the median angular resolution from $20.2^\circ$ to $3.2^\circ$. These results suggest that mechanistic interpretability can reveal learned latent physics encoded within a model's internal representation and help design downstream tasks that exploit it.
Read the full abstract ▾
All authors Raphaël Bonnet-Guerrini, Johann Ioannou-Nikolaides, Inar Timiryasov, Vincenzo Piuri
Artificial Intelligence
Planetary Prediction Engine: Autonomous Geospatial Prediction via Intelligent Data Selection and Foundation Model Embeddings
Evelyn Ma, Rama Kumar Pasumarthi, Kishwar Shafin et al. (+26 more)
Addressing critical global challenges, from food security and disaster risk to disease outbreaks and socio-economic vulnerability, demands high-fidelity geospatial modeling. However, building predictive planetary models remains bottlenecked by a fragmented data ecosystem, requiring manual data
retrieval,
multimodal data curation and fusion along with iterative model selection. We present the Planetary Prediction Engine (PPE), an autonomous AI system that executes this end-to-end workflow directly from natural-
language queries. PPE synthesizes
multimodal datasets on the fly, retrieving spatiotemporally relevant
covariates across open-web and Earth observation platforms (Data Commons, Google Earth Engine) and fusing them with geospatial foundation model
embeddings (PDFM, AlphaEarth). Simultaneously, it searches over task-tailored model architecture families with automated overfitting guards. Across diverse tasks, geographies, and scientific domains, PPE consistently outperforms state-of-the-art or manually tuned expert baselines. For US spatial regression, PPE improves mean $R^2$ across 21 CDC health indicators (76.8% vs. 60.0%), FEMA national risk indices (64.9% vs. 60.0%), and the Social Vulnerability Index (66.2% vs. 58.6%). For spatial downscaling in data-scarce settings, PPE integrates localized proxies to double baseline accuracy in Nigerian food security indicators ($R^2$ of 66.1% vs. 31.5%). For epidemiological nowcasting of the 2026 DRC Bundibugyo Ebola outbreak, PPE achieves a Recall@10 of 83.3% (identifying 15 of 18 newly invaded health zones across five weekly forecasts), a +10.3 percentage-point improvement over the public state-of-the-art modeling (~73%). By combining autonomous
multimodal planetary data discovery with targeted model optimization, PPE lowers the technical barrier to planetary-scale analytics, enabling rapid, customized, expert-level deployment.
Read the full abstract ▾
All authors Evelyn Ma, Rama Kumar Pasumarthi, Kishwar Shafin, Mandar Sharma, Mimi Sun, Hamed Sadeghi, Dav M. Ebengo, Mbulayi Onesime, Ciara Judge, Rouslan Solomakhin, John Wamburu, William Ogallo, Aisha Walcott-Bryant, Sanxing Chen, Arbaaz Muslim, Yael Mayer, Ronald Ho, Roy Lee, Ruth Alcantara, Abdoulaye Diack, Monica Bharel, Lambert Rosique, Jeremy Amez-Droz, Christopher Haire, James Manyika, Yossi Matias, Niv Efron, Gautam Prasad, Shravya Shetty
Artificial Intelligence · Cross-listed from cs.LG
TraceML: What Auto-Research Agents Miss in Long-Horizon ML Development
Jiarui Yan, Weiwei Sun, Sijie Li et al. (+2 more)
Auto-research
agents now run machine-learning development unattended for hours, revising data pipelines, models, and validation from their own feedback, yet on most competitions they still finish below strong human competitors. Outcome-based
benchmarks record this gap but not its cause, because they grade the final submission and discard the development process behind it. We introduce TraceML, which pairs human and
agent work on the same competitions under one version-level schema: 4{,}465 human Kaggle trajectories across 134 competitions, seven of which are also worked by two
agent scaffolds, giving 430 paired human and 207
agent trajectories. Every code version carries its score, its timestamp, and labels for the action taken, its intent, the edit size, and the score effect. Read this way, the gap becomes concrete. Experts alternate data work, validation, model changes, and ensembling, and return to approaches they had set aside. Each
agent scaffold instead collapses into a narrow loop: Codex spends its steps re-weighting ensembles and tuning submissions, MLEvolve mutates its model in place, and neither pivots at the human rate nor reopens abandoned work. A short planning
prompt distilled from human practice moves the behaviors it names toward the human profile and lifts scores, but the effort profile stays
agent-shaped: instruction closes only the part of the gap that reduces to instructions. We release the corpus, the schema, and the labeling models, together with an open-source toolkit that turns a run from any command-line
agent into a TraceML trajectory and reads it against the human cohorts.
Read the full abstract ▾
All authors Jiarui Yan, Weiwei Sun, Sijie Li, Wenhan Li, Yiming Yang
Artificial Intelligence · Cross-listed from cs.LG
ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on
Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer et al. (+7 more)
Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Before deployment, models should therefore be audited for reliance on concepts such as acquisition artifacts and patient demographics. Current methods, including linear probes and concept activation vectors, measure concept reliance by asking whether each concept is decodable from a layer. We show that this framing is misleading: decodability reflects not only concept reliance but also correlations among concepts and the outcome in the
auditing dataset. We reframe shortcut
auditing as a decomposition of the variance of intermediary representations with Independent Canonical cONcepts (ICON). ICON quantifies the share of a layer's variance explained by each concept conditional on all other concepts and the outcome, and also reports the variance left unexplained. Because ICON explanations are comparable across layers, their trajectories distinguish concepts that drive predictions from those that are merely encoded in the representations. In simulated data with ground truth, ICON recovers concept importance more accurately than seven baselines. In skin-cancer and neuroimaging applications, it separates learned shortcuts from merely correlated concepts, which we confirm with independent validation checks.
Read the full abstract ▾
All authors Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer, Marc-Andre Schulz, Nys Tjade Siegel, Maximilian Dreyer, Frederik Pahde, Wojciech Samek, Sonja Greven, Kerstin Ritter
Artificial Intelligence
SwarmWorld: Stigmergic technological evolution in societies of language-model agents
Subhadeep Pal, Fiona Y. Wang and Markus J. Buehler
Collective intelligence can emerge when individuals coordinate through a shared environment, allowing local actions to accumulate into durable social organization.
Language-model
agents offer a new substrate for this process, yet most multi-
agent systems rely on direct conversation, predefined roles, or centralized workflows. It remains unclear whether decentralized
agents can build functional technologies and outperform independent search. Here, initially homogeneous LLM
agents in SwarmWorld self-organize without assigned roles or recipes into evolving technological societies.
Agents explore a spatial environment, process resources, test materials, construct persistent artifacts, and write executable controllers evaluated by a deterministic simulator under unseen disturbances after the
agents are removed. SwarmWorld splits cognition from consequence:
agents propose architectures and controllers within fixed action and material schemas, while the simulated world determines function. Shared societies develop broader, more resilient technological portfolios than a strong best-of-N isolated-search baseline, although isolated search remains competitive for the strongest artifact.
Agents differentiate into exploration, construction, maintenance, and coordination behaviors, transitioning as the world matures. Technologies accumulate through collaborative construction, executable inheritance, and persistent
agent-artifact networks, with most reuse beginning through physical observation rather than communication. Explicit cultural mechanisms amplify collaboration and organization, but functional benefits depend on outcome and timescale. Physical stigmergy alone supports capable societies, while interaction drives persistent technological ecologies rather than universally superior individual inventions.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
Gating Before Commitment: Anticipating Intent Divergence to Prevent Post-Interaction Decision Failures in Autonomous Driving
Cong Xu and Ravi Sankar
Intent misinterpretation during vehicle interactions causes recurring planning failures. We study a decision layer in which a
language-guided intent module reads structured descriptors, computes a smoothed intent-geometry divergence score, and gates the planned maneuver before commitment, upstream of a corridor envelope. On a replayed off-road departure and four crash clips under a frozen, disclosed implementation, gating is the only layer that repairs the plan: on the main case it fires 72 ms after the drift onset but 161 ms before the corridor exit, keeping the trajectory in the corridor in all ten replays. The first calibration draws nine false triggers in 5.9 minutes, each from scoring uncertainty as half a conflict; a preregistered redesign treating uncertainty as abstention cuts this to 0.341 per minute. Two ablations bound the model's contribution: the full score detects fastest on four of five failures under the deployed eligibility, three of five against the unvetoed rule (000871 by one cycle; 000228 by a pre-onset fire on an uncertain stretch that five clips cannot classify as signal or coincidence; dropping the confidence term costs two detections), while on in-domain tracks at equal false positives the
geometric rule more than triples its detection. The evidence supports the gating mechanism; the model's demonstrated roles are the fastest detection on these failures and an uncertainty veto on the
geometric rule.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Prefix Sliding for efficient test-time scaling
Niklas Muennighoff, Zhengyang Wang, Zeyi Chen et al. (+15 more)
Test-time scaling uses extra test-time compute to improve performance, such as letting
language models reason longer when solving a problem. As models keep the entire
reasoning trace in
memory via full
attention, hard tasks that need long thinking can be prohibitively expensive. However, we find most intermediate
reasoning tokens lose importance as the model continues
reasoning. This calls into question whether retaining them is worth the cost. Based on this insight, we propose Prefix Sliding, which discards
tokens during
reasoning that are not part of the prefix or the window of the last few thousand
tokens. The prefix has key instructions and tools available to the model, while the most recent
tokens are the current
reasoning the model is working on. This caps the total
memory requirement regardless of how long the model reasons, allowing for efficient long-horizon test-time scaling. Without training, Prefix Sliding can make existing models 3x faster while maintaining performance. Training with Prefix Sliding using reinforcement learning can achieve better performance by enabling scaling to
reasoning traces beyond a hundred thousand
tokens. Ablations show Prefix Sliding outperforms summarizing intermediate
tokens or vanilla sliding window. Our code is at https://github.com/Muennighoff/prefix-sliding
Read the full abstract ▾
All authors Niklas Muennighoff, Zhengyang Wang, Zeyi Chen, Weijia Shi, Binyuan Hui, John Yang, Dapeng Jiang, Mika Senghaas, Fares Obeid, Johannes Hagemann, Sami Jaghouar, Ludwig Schmidt, Percy Liang, Jason Wei, Andrew Y. Ng, Luke Zettlemoyer, Yejin Choi, Mike Lewis
Artificial Intelligence · Cross-listed from cs.RO
$R^3$: Training Robots to Reason in Natural Language via Reinforcement Learning
Lehong Wu, Yuxiao Qu, Zheyuan Hu et al. (+4 more)
Reasoning in
language allows foundation models to spend more test-time compute on hard problems, such as those requiring decomposition, constraint tracking, and prediction of future consequences. Whether this mechanism can improve robotic manipulation remains unclear, where long-horizon tasks require tracking partial progress,
reasoning about object relations, recovering from mistakes, and steering noisy low-level policies. In this paper, we study whether VLMs can be trained to reason directly in natural
language to guide low-level manipulation policies. We introduce $R^3$, a simple post-training recipe that turns off-the-shelf VLMs into robotic reasoners: it first mid-trains a VLM on expert-generated
reasoning traces to initialize the desired
reasoning style, then improves the reasoner with single-step rubric-based RL from offline action data. Unlike prior robotic
reasoning methods that mostly use structured traces as auxiliary supervision, $R^3$ trains free-form
language reasoning to produce test-time guidance for action. We instantiate $R^3$ on
Language Table and simulated bimanual grocery packing, two controlled testbeds for studying robotic
reasoning and long-horizon manipulation. $R^3$ improves exploration and generalization across unseen tasks and significantly outperforms instruction-only imitation learning baselines on both
benchmarks. Our analyses suggest that free-form
language reasoning can function as a test-time compute mechanism for steering low-level policies. Our project page is available at https://robotic-reasoner.github.io/.
Read the full abstract ▾
All authors Lehong Wu, Yuxiao Qu, Zheyuan Hu, Ivan Zhang, Limin Wei, Zackory Erickson, Aviral Kumar
Artificial Intelligence · Cross-listed from cs.LG
How Much Rank Does LoRA Need? Rank-Error Bounds for Transformer Attention
Gerard Conangla Planes
Choosing the rank of a low-rank adaptation (LoRA) update is usually an empirical task. In this paper, we provide a task-dependent theory of the approximation error achievable at each LoRA rank for
Transformer attention. We fix a pretrained
attention head, a target
attention function, and a distribution over inputs from the downstream task, and bound the smallest expected Kullback–Leibler (KL) error achievable by a rank-$r$ query LoRA update. When target
attention probabilities are bounded away from zero, we prove a lower bound of the error proportional to $\psi(\|d\|_2)$, where $d$ is the difference between candidate and target
attention scores and $\psi(t)=\min\{t^2,t\}$. We also prove an unconditional upper bound $\min\{\|d\|_2^2/4,\sqrt2\|d\|_2\}$. Under explicit realizability, geometry, and moment conditions, we then bound the best rank-$r$ error between an explicit multiple of $\psi(\sqrt{T_r})$ and $\min\{T_r/4,\sqrt{2T_r}\}$, where $T_r$ is the downstream-weighted tail energy of the target update. We also provide target-Fisher bounds when candidate scores remain within a fixed range of the target scores, and an unrestricted lower bound when a subset of
tokens carries most of the probability mass. These spectral bounds describe finite-score approximation. We then construct explicit families in which softmax saturation makes the rank required to match the
attention function strictly smaller than the rank required to match the finite logits. Finally, we extend the analysis to fused multi-head LoRA and joint query/key updates, exposing the effects of rank sharing and query/key factorization constraints.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from math.OC
The Value of Human Expertise
Bradley Sturt
We consider optimization applications with unknown parameters where the decision maker believes that the optimal value of the nominal problem-the optimization problem they would have solved if the true parameters were known-is unlikely to be large. This belief derives from information that humans have that is not captured in
datasets, obtained from domain knowledge and interacting with the physical world. We propose an approach to evaluating policies that provides tighter performance guarantees if the decision maker's belief happens to be correct. Our main result shows that if computing a policy's worst-case performance is a convex program, then the value of human expertise-the maximum improvement in performance guarantees that can be obtained from the belief about the nominal problem-is equal to the minimax gap of a max-min problem. We illustrate our developments in assortment optimization and shortest path problems.
Read the full abstract ▾
Artificial Intelligence
Trace Integrity for LLM Data Agents: Auditing Structured-Data Computation Beyond Answer Accuracy in Real-World Systems
Srimonti Dutta and Akshata Kishore Moharir
Large
language model (LLM) data
agents answer structured-data questions by generating SQL, invoking tools, executing queries, and summarizing results. In operational settings, reliability depends on both reported answer and computation used to produce it, as errors may remain undetected when the resulting answer matches an expected value. We refer to the mismatch between natural-
language intent and operator-level computation required to satisfy that intent as the Structure Gap, and define Trace Integrity as a reliability criterion for assessing whether the computation supporting an answer is explicit, executable, schema-valid, operator-faithful, replayable, answer-consistent, and auditable. We operationalize Trace Integrity through execution contracts that record the schema elements, operator plan, assumptions, executable query, verification status, and final-answer linkage associated with a request. We also define the Correct Answer / Invalid Trace (CAIT) Rate, which measures the proportion of correct answers whose recorded computations fail trace-integrity checks. These provide a way to evaluate whether answer-level success is supported by a valid and inspectable computation. We study this on 100 BIRD Mini-Dev text-to-SQL examples using three prompting conditions: Direct SQL, Operation Summary + SQL, and Contract-First SQL. The three conditions achieve answer accuracies 20.0%, 22.0%, and 24.0%, Trace Integrity Pass Rates 39.0%, 43.0%, and 40.0%, and CAIT Rates 55.0%, 59.1%, and 45.8%, respectively. The results show that answer accuracy, trace validity, and the frequency of correct answers supported by invalid traces capture different aspects of system behavior. Trace-level evaluation therefore complements conventional answer-based evaluation by providing evidence about the computational support, inspectability, and reproducibility of LLM data-
agent outputs in real-world systems.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
DualOPSD: Closing the Feedback Loop in On-Policy Self-Distillation
Yutong Chen, Guangfu Guo, Zhichao Xu et al. (+1 more)
On-policy self-
distillation (OPSD) combines student generated trajectories with dense supervision from a privileged teacher of the same model. Its pointwise clipping protects the student from locally incompatible teacher directions, but the fixed teacher receives no feedback about the supervision the student rejects, which limits effective learning from the teacher. We introduce DualOPSD, which alternates updates in a coupled teacher-student system. The student retains OPSD's clipped forward KL objective. The privileged teacher adapts to the updated student after each student step. A residual target based on the frozen initial model preserves the effect of privileged context during teacher adaptation. Across Qwen3-8B/4B/1.7B and three math
benchmarks, DualOPSD improves avg@12 accuracy over OPSD by 9.83 percentage points. Further diagnostic analyses show lower bidirectional KL, less persistent mismatch, and retained sensitivity to privileged reference. Together, these results demonstrate that DualOPSD can be a more effective form of privileged self-
distillation. Code is available at: https://github.com/on1262/DualOPSD.
Read the full abstract ▾
All authors Yutong Chen, Guangfu Guo, Zhichao Xu, Kunpeng Liu
Artificial Intelligence
Imitation Learning for Connection-Tableau Construction
Fredrik Rømming, Mantas Bakšys, Martin S. Fixman et al. (+1 more)
An automated
theorem prover builds a proof step by step, choosing at each point what to add and what to remove. We cast this construction as a policy acting in a transition system induced by a formal calculus, which fixes which steps are sound: for clausal connection tableaux, leanCoP-style search and plCoP/rlCoP-style planning then become stateful policies over one interface, and policy-learning methods apply directly. We equip such policies with a graph neural network that scores proof edits from structure that transfers across problems, train it by imitation learning from found proofs, and measure how performance holds as we remove search scaffolding, from full symbolic backtracking to a policy the network drives alone. Within a fixed step budget on M2k, MPTP2078-bushy, and TPTP v9.2.1, learned policies solve up to 46% more problems than leanCoP, and reach proofs in an order of magnitude fewer steps.
Read the full abstract ▾
All authors Fredrik Rømming, Mantas Bakšys, Martin S. Fixman, Sean B. Holden
Artificial Intelligence · Cross-listed from eess.AS
VoiceMem: Streaming Dual-Brain Memory for Real-Time Interaction
Zhifei Xie, Jiaqi Lang, Ze An et al. (+7 more)
Conversational systems, such as duplex
speech language models (SLMs), still lack a streaming, accurate, and empathetic
memory system as their soul. We introduce VoiceMem, a simple
memory architecture with a parallel informational left
brain, an emotional right
brain, and streaming
memory I/O mechanisms. We further build a complete pipeline for
memory-aware SLM training, long-horizon evaluation, and decoupled deployment with interchangeable
memory backends. Experiments and real-world deployment show three advantages: i) Accuracy: under top-5
retrieval, the left
brain outperforms classical systems such as Mem0 at top-200 by nearly 30 points; ii) Emotional & Personal: the right
brain, with short- and long-horizon affective attribution and dual-node persona modeling, achieves state-of-the-art performance across three persona
benchmarks and improves the aggregate score by 4.29 points over the previous best system; and iii) Real-Time & Cheap: VoiceMem completes
retrieval in 134 ms, well within standard VAD latency, adding no extra conversational delay while maintaining high accuracy and low cost. These results show that VoiceMem provides a practical
memory foundation for real-time, personalized, and emotionally aware
speech interaction.
Read the full abstract ▾
All authors Zhifei Xie, Jiaqi Lang, Ze An, Yifan Zhao, Dongchao Yang, Kai Li, Ziyang Ma, Mingbao Lin, Chunyan Miao, Shuicheng Yan
Artificial Intelligence
AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs
Sheng Liang, Yongyue Zhang, Nathanael Brian et al. (+4 more)
Agentic LLM pipelines face escalating inference costs as context accumulates across
retrieval, tool use, and multi-turn interactions. To control latency, deployments routinely compress inputs, but this degrades task accuracy. Speculative decoding (SD) accelerates generation losslessly, yet it assumes the drafter and verifier share an identical context, preventing SD from resolving the accuracy-overhead trade-off. We propose AsymSpec, an asymmetric speculative decoding framework that breaks this
symmetry: a lightweight drafter reads the full input while the large verifier operates on the compressed view. The drafter steers the verifier via a contrastive $\delta$-fusion of logits, modulated by a divergence-aware acceptance gate that preserves verification stability and high draft acceptance rates. Evaluated across four agentic capabilities and two end-to-end
agent benchmarks, AsymSpec reaches $\approx 90\%$ of full-context accuracy on average, delivering $1.3$–$1.7\times$ throughput speedups at $0.2$–$0.3\times$ the compute cost on isolated text capabilities. These results show that asymmetric context access yields substantial gains precisely when compression discards critical
reasoning signals.
Read the full abstract ▾
All authors Sheng Liang, Yongyue Zhang, Nathanael Brian, Hang Lv, Hao Wang, Chen Zhang, Yong Liu
Artificial Intelligence
ProgRouter: Online Progress-Guided Orchestration for Multi-Agent LLM Workflows under Quality-Cost Tradeoffs
Songyuan Li, Ahmed M. Abdelmoniem and Shiqiang Wang
Multi-
agent large
language model (LLM) workflows have emerged as a powerful paradigm for solving complex, open-ended tasks through collaborative
reasoning among specialized LLM
agents, but they incur substantial operating costs due to repeated LLM invocations and long-horizon context accumulation. Existing cascade routing methods make one-shot, query-level decisions and cannot adapt to the dynamic, state-dependent nature of multi-step workflows, in which the right LLM at each step depends on evolving task progress, remaining task difficulty, and cost-efficiency requirements. We present ProgRouter, an online progress-guided routing framework that adaptively selects LLM
agents across workflow steps to preserve task-solving quality while adhering to time and cost budgets. ProgRouter introduces a multi-view task progress scorer that combines coarse workflow outcome regimes with fine-grained signals on subtask completion, progress trends, and workflow state quality. Then, a dual-path task progress predictor and an adaptive meta-gating mechanism estimate the progress gain for each candidate routed LLM. ProgRouter makes online step-wise routing decisions that balance progress gain, task time budgets, and long-term operating cost efficiency. Experiments on HumanEval Plus, MBPP, MATH-500, and ASQA, spanning agentic code generation, mathematical
reasoning, and
retrieval-augmented long-form question answering, demonstrate that ProgRouter reduces the operating cost relative to key baselines while maintaining strong task-solving performance.
Read the full abstract ▾
Artificial Intelligence
Multi-Granularity Context-Enhanced RAG over Multimodal Knowledge Graphs
Zongyu Wu, Yilong Wang, Xiaochen Wang et al. (+5 more)
Retrieval-augmented generation (RAG) is widely used to mitigate
hallucination issues in large
language models (LLMs) and
multimodal large
language models (MLLMs). In particular, knowledge graph (KG)-based RAG leverages structured knowledge to provide (M)LLMs with high-quality external information. Building on these works, recent studies have explored
multimodal knowledge graphs (MMKGs) as knowledge bases for GraphRAG. This enables Graph RAG to integrate knowledge across multiple modalities, thereby further enhancing its performance. However, existing MMKG-based RAG methods generally follow a common pipeline in which different modalities are largely processed independently before being fusion. As a result, textual context is only used to a limited extent during visual information extraction and subsequent
multimodal knowledge fusion. This brings a semantic gap between images and text which limits the
multimodal GraphRAG performance. To address this issue, we propose a novel framework for constructing a Context-Enhanced MMKG (CEMMKG) to better support
multimodal GraphRAG. The proposed CEMMKG enriches each image with complementary textual context at both local and global scopes. Local context goes beyond the surrounding text by incorporating sentences that are semantically related to the image, while global context provides a summary of the entire passage. We further introduce a multi-granularity design for the local context, allowing it to capture semantically relevant information at different levels of detail. Extensive experiments on the selected
vision-centric
dataset validate that CEMMKG is effective in leveraging contextual information to improve MMKG-based RAG performance. Moreover, its effectiveness across different MMKG-based RAG methods demonstrates its broad applicability.
Read the full abstract ▾
All authors Zongyu Wu, Yilong Wang, Xiaochen Wang, Minhua Lin, Zhichao Xu, Fenglong Ma, Xiang Zhang, Suhang Wang
Artificial Intelligence · Cross-listed from cs.CV
FRAME: Separating sampling variation from performance disparities in medical image analysis
Mahshad Lotfinia, Daniel Truhn, Andreas Maier et al. (+1 more)
Fairness audits of medical imaging models commonly report the largest performance difference between demographic subgroups and treat any positive value as evidence of bias. Yet a perfectly fair model also produces a positive difference, which grows as subgroups shrink. Some mitigation methods remove demographic information from models because they assume that it produces the difference. Here we introduce Fair-model Reference And Mechanism Evaluation (FRAME). Its first step compares each reported difference with the difference expected from a perfectly fair model at the same subgroup counts. Its second step tests whether candidate causes of any excess over this fair-model reference change the difference when they are injected into model features. We evaluated 36 encoders on nine
datasets in three modalities and audited 89 published differences across six modalities. For 10 frozen encoders and 13 chest radiograph findings, the reference is a median 41% of the race difference and 22% of the age difference. It is exceeded by 40 of 53 published differences in sensitivity or false positive rate and by one of 36 in the area under the receiver operating characteristic curve (AUROC). Adding race information to the features of two encoders does not change the race difference significantly. At matched disease performance, mitigation reduces the race difference by less than its spread across three pretraining seeds (medians 0.005 and 0.012) and the age difference by more (0.008 and 0.006). Image-text pretraining raises the worst-group AUROC for race by a median 0.049 over self-supervised pretraining. Reporting the fair-model reference beside every new and published subgroup difference could direct mitigation to the differences above it.
Read the full abstract ▾
All authors Mahshad Lotfinia, Daniel Truhn, Andreas Maier, Soroosh Tayebi Arasteh
Artificial Intelligence
SciMIF: Understanding Multimodal Instruction Following in Scientific Domains
Ye Shen, Yuting Zheng, Dun Pei et al. (+4 more)
Understanding instruction-following capabilities in scientific domains is essential for effectively leveraging
Multimodal Large
Language Models (MLLMs) to advance the development of scientific fields. In this work, we introduce SciMIF, a novel
benchmark designed to evaluate the capability of MLLMs in following complex scientific instructions. Specifically, based on an extensive analysis of 22 distinct tasks across 5 representative scientific disciplines, we propose a comprehensive taxonomy comprising 10 constraint groups that captures both general functional requirements and discipline-specific characteristics. Guided by this taxonomy, we develop a high-fidelity instruction injection pipeline to systematically augment existing scientific
datasets. We conduct comprehensive experiments on multiple state-of-the-art closed-source and open-source MLLMs. Our findings reveal significant performance disparities across different scientific disciplines, with chemistry posing greater challenges for current MLLMs. Furthermore, we observe that increasing the model scale does not yield corresponding improvements in constraint adherence, and current models still struggle severely with fine-grained constraints and instructions requiring the deep application of disciplinary knowledge. SciMIF fills the current void in evaluating
multimodal instruction adherence within scientific domains, laying a crucial foundation for future enhancements of MLLMs in rigorous scientific applications. Data and code will be released at https://github.com/shenye7436/SciMIF .
Read the full abstract ▾
All authors Ye Shen, Yuting Zheng, Dun Pei, Zijian Chen, Wenlong Zhang, Qi Jia, Guangtao Zhai
Artificial Intelligence · Cross-listed from cs.CV
PANDA - Prototype-Anchored Alignment for Partially Unpaired Multimodal Learning, with Applications to Alzheimers MRI and TCGA Pathology
Sheethal Bhat, Mahfuzur Rahman Chowdhury, Paula Andrea Perez-Toro et al. (+4 more)
Multimodal medical prediction often faces incomplete pairing: auxiliary modalities with complementary signal are available for only a subset of subjects (or none) and cannot be assumed at deployment. We introduce PANDA (Prototype Anchored Data
Alignment), a two-stage framework that transfers auxiliary information to a primary-modality model without auxiliary inputs at inference. Stage 1 learns a shared
embedding from the paired subset and estimates class prototypes from auxiliary modalities; Stage 2 trains the primary encoder on all subjects using cross-
entropy plus
alignment to the frozen prototypes. Because supervision is defined at the class-prototype level, PANDA accommodates arbitrary pairing rates, including zero subject overlap. We evaluate PANDA on two applications. On a 1,021-subject multi-scanner ADNI cohort, we perform AD/CN classification with three auxiliary modalities at distinct pairing rates: tabular scores (44.8%), FDG-PET (18.7%), and external handwriting kinematics (0% overlap). Relative to the same-backbone MRI-only baseline, PANDA attains AUC 0.868 +-0.020 (+7.9pp) and reduces 1.5T CN false positives by 24.3pp; on a fully trainable Conv5-FC3 backbone it reaches AUC 0.893 (best overall). A pairing-rate ablation shows that the joint anchor remains within seed noise from 75% to 5% pairing. On TCGA-Lung survival prediction from whole-slide images with RNA-seq as auxiliary data, PANDA improves over WSI-only on 2-year OS (AUC +3.5pp) and Cox PH (C-index +9.0pts) and outperforms full-fusion training, which underperforms WSI-only, while requiring no RNA at inference; wide confidence intervals on this smaller cohort keep the gains below conventional significance. Overall, PANDA provides a deployment-oriented mechanism for leveraging incomplete auxiliary modalities to improve primary-modality prediction.
Read the full abstract ▾
All authors Sheethal Bhat, Mahfuzur Rahman Chowdhury, Paula Andrea Perez-Toro, Stephan Wunderlich, Rose Dawn Bharat, Siming Bayer, Andreas Maier
Artificial Intelligence
Quantitative Analysis of $\omega$-Regular Robust MDPs
Ali Asadi, Krishnendu Chatterjee, Ehsan Kafshdar Goharshady et al. (+3 more)
Robust Markov Decision Processes (RMDPs) generalize classical MDPs by allowing uncertainty in transition probabilities and optimizing against their worst-case realization. We consider $(s,a)$-rectangular RMDPs with linearly defined uncertainty sets and study parity objectives, which are a canonical representation of $\omega$-regular objectives. An uncertainty set is linearly defined if it is described by linear inequalities over the transition distribution together with auxiliary variables, which capture the standard $L_1$ and $L_\infty$ balls as well as general polytopic uncertainty sets. The quantitative value is the supremum, over all
agent policies, of the satisfaction probability guaranteed against the
adversarial environment. Previous work studied the qualitative analysis, namely the almost-sure (resp. positive) problem that asks whether a single
agent policy guarantees satisfaction with probability one (resp. positive probability) against every environment policy. In this work, we solve the exact quantitative problem. Our contributions are threefold. First, we show that both the
agent and the environment admit pure memoryless optimal policies. Second, we give a
polynomial-time
algorithm for quantitative parity on linearly defined robust Markov chains and use it as a subroutine in a policy-iteration
algorithm for RMDPs. The
algorithm combines quantitative one-step improvements with qualitative almost-sure improvements. Finally, we report experiments comparing our approach with the explicit reduction to
stochastic games.
Read the full abstract ▾
All authors Ali Asadi, Krishnendu Chatterjee, Ehsan Kafshdar Goharshady, Mehrdad Karrabi, Alipasha Montaseri, Ali Shafiee
Artificial Intelligence
LivingRAG: Augmenting Graph RAG with Experience
Yuzhuo Cui, Zongye Zhang and Qingjie Liu
Graph-based RAG improves multi-hop question answering by organizing evidence as a knowledge graph. However, most existing RAG systems process each query in isolation and discard useful
reasoning from the LLM's response after inference. As a result, later related queries need to retrieve evidence and reason from scratch. We propose LivingRAG, a Graph RAG framework with writable and reusable
reasoning experience. LivingRAG adds a writable experience store to a graph-based
retrieval backbone, enabling verified experiences to be reused during inference in two ways. Stored graph signals help
retrieval find entities and passages that were useful in earlier related queries. Stored summaries provide a reference
reasoning pattern for answer generation. We analyze online QA streams and find reusable signals from shared entities, graph neighborhoods, and question templates. Experiments on multi-hop QA
benchmarks show that LivingRAG improves accuracy over strong RAG baselines and reduces completion-
token use when relevant prior experience is reused.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
A Statistical Audit of Physical AI Benchmark Redundancy
Zaruhi Navasardyan and Hrant Davtyan
Physical AI models are evaluated on suites of
benchmarks that differ across model reports, leaving the model-by-
benchmark matrix sparse and the relationship between
benchmarks unmeasured. We construct a
matrix of 51 models on 12 physical AI
benchmarks, selected from a registry of 51
benchmarks and 152 models by reporting density, combining scores from model cards and
benchmark papers with our own evaluation runs under each
benchmark's official protocol. We measure how much information the
benchmarks share and show quantitative evidence of Redundancy. Redundancy affects reported rankings: collapsing the two substitute pairs into single columns moves 22 of 51 models by three or more places under an equally weighted average. We then select
benchmarks greedily under a utility combining score dispersion with variance not explained by the already-selected set, and obtain a four-
benchmark subset retaining 78.5% of the utility of all 12, on which we fit a Bradley–Terry ranking. The procedure requires only
benchmark-level scores with sufficient overlap and is not specific to physical AI.
Read the full abstract ▾
Artificial Intelligence
Candidate supply and answer selection shape the value of LLM judging in multi-agent systems
Jia-Hao Ji, Sijie Li, Jiabei Cheng et al. (+3 more)
Multi-
agent systems (MAS) sometimes already have the potential to answer correctly, but still report a wrong answer. Explaining this outcome is difficult because generation, communication and final answer-selection rules usually change simultaneously. We conceptualize multi-
agent reasoning as an evolutionary pipeline of candidate generation, peer communication and terminal selection, wherein consensus without quality control can exhibit patterns of memetic drift. We study two questions: (1) when an LLM judge provides effective selection pressure by supplying a signal of answer correctness for candidates generated in a multi-
agent system, and (2) when using that signal improves the reported answer. To map judge reliability, we analysed 15,336 questions from MMLU-Pro, GPQA, MedXpertQA and MuSR, with Humanity's Last Exam analysed separately. To test these rules, we replayed 81,390 fixed candidate pools drawn from 16,278 questions across five
benchmarks. We report three findings. (1) A correct answer is often already present among the generated candidates, but the system can still converge on and report a wrong answer. (2) Judge reliability is not a fixed trait of the model, but varies with the task, the generator and how rare the correct answer is. (3) Combining answer frequency with the judge's evaluation changed only the final answer-selection rule and raised accuracy from 63.82% to 70.82-70.95%, primarily by rescuing correct answers that were outnumbered by popular errors. In the systems studied here, the value of generating more candidates depends on whether those extra samples make correct answers present, frequent or recognisable. By isolating generation, recognition and selection, these findings establish a diagnostic basis for designing multi-
agent architectures that protect generated correct answers from being lost.
Read the full abstract ▾
All authors Jia-Hao Ji, Sijie Li, Jiabei Cheng, Zixi She, Jin-Tai Yu, Zhiyuan Yuan
Artificial Intelligence · Cross-listed from cs.LG
One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation
Justin Robert and Raheel Qader
On-policy
distillation trains a
language model on its own generations while a teacher scores them
token by
token. It combines the dense supervision of imitation learning with the on-policy sampling of reinforcement learning. But it requires a second, larger model to act as teacher. On-Policy Self-
Distillation (OPSD) removes that cost. The teacher is the model itself, conditioned on privileged information the student will not have at test time, such as a reference solution, a plan, or environment feedback. The teacher is no stronger than the student, only better informed. Early results were promising, with accuracy comparable to reinforcement learning at a fraction of the generated
tokens. But the same asymmetry that produces the signal also biases it. One failure mode now dominates the field: collapse, the progressive narrowing of the set of
reasoning paths the model can produce. Collapse is not specific to OPSD, though privileged information aggravates it. This review treats collapse as a symptom governed by three levers: (i) where the signal is applied, that is, how
tokens are weighted; (ii) what the teacher is shown, that is, the nature of the privileged information; and (iii) when the signal changes, that is, the teacher's dynamics and the decay of guidance. We restrict our scope to mathematical
reasoning, where the method originated and where its failure modes are best documented. We report no new experiments. The contribution is structural: a shared vocabulary for phenomena named differently across papers, and a clear line between what is settled and what is still disputed.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understanding
Yuqiang Lin, Yan Shi, Sam Lockyer et al. (+5 more)
Traffic Anomaly Understanding (TAU) requires models and systems to detect, reason about, and explain anomalous events in transportation
videos. To address this challenge, we propose TAU-
Agent, an agentic
retrieval-augmented framework for traffic anomaly understanding. Given a task query, a central
retrieval agent orchestrates two visual perception tools, namely a
Video Captioning Tool and an Open-Vocabulary Tracking Tool, to retrieve and select query-relevant evidence, including captions, temporal intervals, and object trajectories. The selected evidence, together with sampled
video frames and the input query, is provided to a supervised fine-tuned
vision-
language model for final
reasoning and answer generation. We evaluate TAU-
Agent on both the in-domain and the out-of-domain
benchmarks from the AI City Challenge 2026. TAU-
Agent achieves scores of 0.6779 on Track 3, 0.3998 on Track 7, and 67.9275 on Track 8, ranking second, twelfth, and fifth, respectively. Code is available at: https://github.com/siri-rouser/TAU-
Agent.
Read the full abstract ▾
All authors Yuqiang Lin, Yan Shi, Sam Lockyer, Harish Tayyar Madabushi, Adrian Evans, Wenbin Li, Yinhai Wang, Nic Zhang
Artificial Intelligence
How Robust Are Automated Fact-Checking Systems? A Cross-Benchmark Evaluation
Aida Usmanova, Zangir Iklassov, Markus Leippold et al. (+1 more)
Automated fact-checking (AFC) systems retrieve evidence and predict claim veracity, yet evaluations omit simple baselines, systems are developed for a single
benchmark and cannot be trusted to generalise across domains. No prior work cross-evaluates the full two-stage retrieve-then-verify pipeline across diverse
datasets, complementing
retrieval-only studies (Thakur et al., 2021) and single-stage benchmarking studies (Calamai et al., 2025). We
benchmark nine models, ranging from random and sparse baselines to fine-tuned
transformers, zero-shot LLMs, and the two highest-ranked systems from the AVeriTeC 2025 shared task, across four
datasets spanning scientific, open-web, and
climate domains. Three findings stand out: (1) on ClimateCheck claim-only and fine-tuned models outperform zero-shot LLM and top-performing AVeriTeC 2025 systems, highlighting that noisy evidence can degrade veracity prediction; (2) system rankings are strongly domain- and metric-dependent: the best model on SciFact (macro-F1 0.70) drops to 0.31 on ClimateCheck, while the AVeriTeC 2025 winner and runner-up swap rankings based on evaluation metrics and
datasets; (3) replacing retrieved evidence with gold annotations improves veracity accuracy by 14-22 points across models, confirming
retrieval remains primary bottleneck. We release code, pre-processed
datasets, and all results to support reproducible AFC research.
Read the full abstract ▾
All authors Aida Usmanova, Zangir Iklassov, Markus Leippold, Ricardo Usbeck
Artificial Intelligence · Cross-listed from cs.CV
When Composition Doesn't Add Up: Humans Identifying Defects in AI-Generated Images
Ruoqi Hu, Chulin Zhao, Jiashuo Chang et al. (+2 more)
*Chulin Zhao and Ruoqi Hu contributed equally to this work. State-of-the-art text-to-image (T2I) models exhibit pronounced and systematic defects when
prompts involve intricate compositional factors such as multiple entities and multiple attributes. In this paper, we investigate how humans identify such defects. Specifically, we manually select 651 reference images from the four categories of people, hand, object, and scene that exhibit complex compositional characteristics, from which
prompts emphasizing compositional factors are derived by manually editing ChatGPT-generated
prompts. We then feed the
prompts into three selected T2I models to generate AI images and conduct a comprehensive subjective study to identify their defects. For each image, 29 participants provide multi-label assessments specifying defect types and locations. The study yields the compositional AI-generated image defect (CO-AID)
dataset, including reference images,
prompts, AI-generated images, and information on defect locations and types. Experimental results show that training a deep model on CO-AID can both predict defects in AI-generated images and optimize AI image generation, demonstrating its usability and effectiveness. The database and supplementary materials are available at: https://github.com/Future-IQA/CO-AID .
Read the full abstract ▾
All authors Ruoqi Hu, Chulin Zhao, Jiashuo Chang, Ramon Ruiz-Dolz, Hanhe Lin
Artificial Intelligence · Cross-listed from cs.CV
Code World Model: Coding Agent as World Brain
Yiwen Chen, Guosheng Lin and Chi Zhang
World models aim to simulate how complex environments evolve under actions and events, yet existing
video-based world models primarily learn dynamics from visual observations, which reveal outcomes rather than the underlying knowledge, rules, and mechanisms governing world
evolution. This makes it difficult to maintain persistent consequences and support coherent, open-ended
evolution. We introduce Code World Model, a framework that separates world
evolution from visual realization by combining the
reasoning and coding capabilities of
language models with the generative priors of
video models. A coding
agent serves as the world
brain,
reasoning about events and their consequences and generating executable code to maintain persistent world state and perform rule-consistent
evolution. To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wise spatiotemporal constraints and is compiled into a proxy
video, which conditions a
video model to render high-fidelity visual observations. We further develop data pipelines for constructing aligned proxy-observation pairs from gameplay and real-world
videos. After fine-tuning on paired gameplay data, MiniMax-H3 follows proxy-based spatiotemporal specifications from simple interactive worlds built by the coding
agent while preserving rich visual details and dynamics. These results demonstrate the potential of combining code for persistent world
evolution with
video models for flexible visual realization, providing a new path toward open-ended world models.
Read the full abstract ▾