Artificial Intelligence · Cross-listed from cs.CV
Rethinking Projector Training in Multimodal LLMs
Fabian Gröger, Naël Ouerghemi, Shuo Wen et al. (+1 more)
Multimodal LLMs are typically trained by first aligning a
vision encoder with a
language model via projector training on large image-caption
datasets, followed by joint training of the full
multimodal model. Despite its widespread use, the role of the projector training stage remains poorly understood. We find that (i) using only $10$ to $20\%$ of the data for training the projector already recovers most downstream performance gains, and (ii) projectors trained on different
datasets can be linearly interpolated while largely preserving performance. These results suggest that projector training mainly performs a coarse
alignment, which many different projectors achieve equally well. Motivated by these findings, we propose PORTAL, a training-free projector initialization that requires neither paired image-caption data nor gradient-based optimization, but only per-modality summary statistics. PORTAL computes an optimal-transport-based map between Gaussian approximations of the
vision and
language feature distributions on a shared principal subspace, so that the projector training stage can be skipped entirely. Without any paired data, PORTAL matches the standard pipeline trained on 558k image-caption pairs across six LLM backbones, two
vision encoders, and 16
benchmarks. It outperforms that pipeline in $3$ of $7$ (
vision encoder, LLM) settings and reaches an average $98\%$ of its score across the other four.
Read the full abstract ▾
All authors Fabian Gröger, Naël Ouerghemi, Shuo Wen, Maria Brbić
Artificial Intelligence · Cross-listed from cs.SD
PhysicsSliders: Learning Physical Controls for Audio Foley Generation
Peiwei Chang, Yongqiang Chen, Yuelan Cheng et al. (+6 more)
Physical properties provide intuitive controls for sounds made by interacting objects. The challenge is to make a generative model respond predictably to these controls, both individually and in combination. We present \method\ (PXS), trained on simulated contact sounds with 15 material, geometry and contact parameters. Two separately trained variants accept direct parameter inputs or numerical captions. PXS controls hardness, size, speed and damping, supports room descriptions and produces spectrally distinct outputs at fixed targets. When controlling damping and size together, specifying thickness improves frequency accuracy and joint target delivery. Measurements from real recordings define effective-damping targets, which simulator inversion maps to generation inputs without real-audio fine-tuning. Across 18 impact classes from five real-audio
datasets, PXS-Physical follows these targets with mean grouped Spearman correlation $\rho=0.976$ and class-median error 0.094 dex (a factor of 1.24).
Read the full abstract ▾
All authors Peiwei Chang, Yongqiang Chen, Yuelan Cheng, Liting Gao, Yishun Lu, Li Zhang, Mark D. Plumbley, Philip J.B. Jackson, Wenwu Wang
Artificial Intelligence · Cross-listed from cs.RO
ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models
Changyeon Kim, Kimin Lee, Yashraj Narang et al. (+1 more)
Reinforcement learning (RL) post-training improves
robot foundation models (RFMs) beyond their demonstrations, but real-world deployment demands more: learning new tasks efficiently while retaining previously acquired behaviors. Existing continual RL methods on top of VLAs mitigate forgetting but rely on on-policy updates with limited forward transfer, whereas off-policy methods improve single-task sample efficiency without addressing learner persistence across tasks. We propose ACCRUE, a continual off-policy RL framework that carries lightweight actor and critic networks across tasks, allowing each stage to reuse behavioral and value knowledge acquired through prior RL. The inherited critic adapts only on current-task transitions and guides current-task actor learning, while prior behavior is preserved by distilling the actor toward cached targets on a small number of retained episodes. Fixed visual features keep the inherited learner and its replay compatible with base-policy updates. Across contact-rich assembly sequences in simulation and on real-
robot hardware, ACCRUE accelerates subsequent-task learning with near-zero forgetting and significantly reduces real-
robot training time by 31.0% to 68.5%.
Read the full abstract ▾
All authors Changyeon Kim, Kimin Lee, Yashraj Narang, Yijie Guo
Artificial Intelligence · Cross-listed from cs.SD
GeoMER: A Joint Euclidean-Hyperbolic Geometry Framework for Multimodal Emotion Recognition
Ting Zhu, Huizhi Liang, Boguslaw Obara et al. (+2 more)
Multimodal Emotion Recognition (MER) aims to understand human emotions by integrating information from multiple modalities such as
speech and text. Psychological studies suggest that emotions exhibit latent hierarchical structured relationships in their semantic representations. However, most MER approaches rely on Euclidean representations, which capture fine-grained cross-modal interactions but do not explicitly provide a
geometric prior for organizing hierarchical emotional structures. To mitigate this gap, we propose GeoMER, a joint Euclidean-
Hyperbolic modeling framework that introduces
hyperbolic emotion-core routing to capture higher-level, abstract emotional representations for MER. The proposed framework jointly models
token-level cross-modal semantic interactions and latent emotion representations in two complementary
geometric spaces. Specifically, GeoMER consists of a Euclidean
Multimodal Fusion Module (EuclFusion) for
multimodal representation fusion and a
Hyperbolic Geometric Core Routing (HypGeoRoute) module that performs geometry-aware core assignment and aggregation in a
hyperbolic manifold to capture more abstract feature representations. The HypGeoRoute module complements the Euclidean
token-level modeling. We experiment on the IEMOCAP
multimodal MER
benchmark and demonstrate that the proposed method achieves the best performance, with 83.54% weighted accuracy and 83.35% unweighted accuracy, outperforming Euclidean baselines, including MemoCMT.
Read the full abstract ▾
All authors Ting Zhu, Huizhi Liang, Boguslaw Obara, Shufei Duan, Varun Ojha
Artificial Intelligence
Compare to Correct: Pairwise-Guided Logit Adjustment for LLM Test-Time Scaling
Cong Hua, Qianqian Xu, Zitai Wang et al. (+4 more)
Sampling-based test-time scaling enhances LLM
reasoning by aggregating multiple
reasoning paths for the same query. Existing methods use answer frequencies or path confidence to compute candidate scores, which do not explicitly characterize comparisons between competing answers. Inspired by the Condorcet principle in social choice theory, we introduce pairwise confidence comparisons between paths supporting different answers to capture answer-level comparative advantage. We combine this comparative evidence with voting support through a KL-regularized objective. Theoretically, we show that this comparative-advantage is equivalent to the adjustment to the traditional voting logits. In this way, our proposed Pairwise-Guided Logit Adjustment (PGLA) framework allows comparative evidence to compensate for weaker voting support and correct voting-based ranking errors. Experiments on 10 challenging
benchmarks including mathematical
reasoning, knowledge
reasoning, and code generation with 7 models ranging from 1.7B to 32B parameters speak to the effectiveness of this plug-and-play term. Additional evaluations with external reward scores and online sampling further demonstrate its applicability.
Read the full abstract ▾
All authors Cong Hua, Qianqian Xu, Zitai Wang, Zhiyong Yang, Shilong Bao, Linchao Zhang, Qingming Huang
Artificial Intelligence
Reading Right, Acting Wrong: Diagnosing Decision Failures of System One Models in Agent Loops
Yubo Li, Ramayya Krishnan and Rema Padman
System One models choose an
agent's next action by scoring a supplied set of options in a single pass. This makes them fast and cheap, but when they choose wrongly there is nothing to inspect: did the model misread what the
agent has seen, or read it correctly and still act wrongly? We answer behaviorally. On a frozen
benchmark of
agent decision states from simulated environments, customer-service policies, tool use, web navigation and coding, we pair every next-action question with factual probes about the same state, each answered independently. Across seventeen fast systems and a
reasoning model, the answer is consistent: most wrong actions follow correct reading, and the more accurate a system, the more of its errors take this form. These errors are not noise; they gather on the same states across systems of every kind. Where the probed facts provably determine the action, models that read every fact still fail to combine them, and their own answers, composed by code, repair the failure. On real states, what is missing is often not a fact but the next sub-goal.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
When More Objectives Make Identification Easier: The Role of Preferences and Feedback
Bo Xue, Ji Cheng, Yuanyu Wan et al. (+1 more)
Best-arm identification (BAI) aims to identify the optimal arm using as few samples as possible. While most existing work focuses on a single objective, many real-world applications involve multiple criteria, which motivates the study of BAI in multi-objective bandits. Intuitively, multi-objective identification seems harder, since the learner must identify optimal arms in a higher-dimensional reward space; however, the additional objectives also provide richer feedback that can reduce the required exploration. In this paper, we characterize when and why multiple objectives can make BAI easier. For Gaussian bandits, we establish a characteristic-time lower bound for fixed-confidence identification that holds uniformly across preference orders. Through explicit comparisons under Pareto, linear, and lexicographic preferences, we reveal three phenomena: (i) identifying a larger optimal set can require fewer samples than identifying a unique optimum; (ii) the advantage of vector feedback over coordinate feedback depends fundamentally on the preference order; and (iii) multi-objective identification can require fewer samples than even the easiest corresponding single-objective problem. Finally, we propose Order-Aware Track-and-Stop (OATS), an
algorithm that is correct at any prescribed confidence level and asymptotically matches the lower bound.
Read the full abstract ▾
All authors Bo Xue, Ji Cheng, Yuanyu Wan, Shuang Qiu
Artificial Intelligence · Cross-listed from cs.CL
The Many Morals of AI: The cautionary bias of North American news narratives in global perspective
Andrew Piper and Edie Pearman
The stories told about artificial intelligence (AI) play a central role in shaping public understanding, policy debates, and expectations about technological change. News media are especially influential in this process. Using a cross-regional corpus of 12,924 English-
language news articles about AI published since 2022, we examine how storytelling about AI varies across media systems in four regions: North America, China, India, and the Middle East. Leveraging large
language models, we extract the implicit narrative lessons conveyed by each article—commonly known as story morals—capturing value-laden interpretations that extend beyond thematic content or sentiment. We find that North American AI coverage is uniquely characterized by narratives of caution, while under-emphasizing narratives of innovation and collaboration that are more prevalent elsewhere. However, this pattern is largely explained by a broader regional negativity bias in North American news sentiment, rather than by heightened concern about AI itself. Our findings bring into view the culturally conditioned nature of AI news narratives.
Read the full abstract ▾
Artificial Intelligence
LLM Agents for Automated Post-Training via Structured Recipe Optimization
Ruohong Liu, Bohan Tang, Lekang Jiang et al. (+2 more)
Automated post-training transforms pre-trained large
language models (LLMs) into models capable of handling diverse domains by automatically constructing and optimizing a recipe of coupled decisions over data sources, data processing, cross-domain composition, and training configuration under a limited experimental budget. Existing automation either searches predefined recipe spaces that enable systematic optimization but remain fixed rather than adapting their available choices to the target tasks, or uses open-ended
agents that discover task-specific choices but leave them implicit in trial-level edits, limiting their systematic reuse and recombination across trials. To fill this gap, we introduce optimization over a task-adapted recipe space, a novel paradigm that can adapt the recipe space itself to the target tasks while representing its choices explicitly for systematic reuse, recombination, and joint optimization. We instantiate this paradigm with Structured Recipe Optimization (SRO), comprising an LLM-based Recipe-Space Designer that constructs the task-adapted recipe space and a Multi-Domain Recipe Optimizer that uses structured trial evidence to guide LLM candidate construction and multi-objective
Bayesian trial selection. Extensive experiments show that SRO achieves the highest average score in all evaluated target-model and
agent-LLM combinations. In joint post-training across three capability domains and six
benchmarks, SRO reaches 41.1 versus 15.4 for the strongest baseline on Qwen3-1.7B with GPT-5.4, and 53.8 versus 40.8 on Qwen3-4B with DeepSeek-V4-Flash. Our code is available at https://github.com/RuohLiuq/SRO.
Read the full abstract ▾
All authors Ruohong Liu, Bohan Tang, Lekang Jiang, Jack Umenberger, Yiwen Guo
Artificial Intelligence
Order-Consistent Latent Geometry for Reliable Short-Horizon Planning in World Models
Junlin Wu, Wen Fan and Dandan Zhang
Latent planners often rank candidate action sequences by terminal Euclidean distance to a goal latent. Exact agreement with transition distance would make this score ideal, but is stronger than planning requires: along a transition-shortest rollout, the essential requirement is that latent distance does not reverse true transition progress. We show that this pathwise no-reversal requirement is precisely the classical self-approaching condition. To the best of our knowledge, we are the first to identify self-approachingness as a planner-facing compatibility condition for terminal Euclidean latent planning and to connect its violation to an exact finite-horizon candidate-order reversal. Because visual world models predict discrete latent sequences, we analyze their polygonal interpolation. We prove that an acute bend creates a nonempty region in which terminal Euclidean distance ranks a transition-worse candidate ahead of one that makes genuine progress, even under exact rollouts. We further interpret latent dimension as angular capacity through a classical spherical-code bound. A controlled six-branch experiment, including a zero-padded three-dimensional baseline, validates the predicted ordering failure and separates angular arrangement from ambient dimensionality.
Read the full abstract ▾
Artificial Intelligence
Configuring RAG Systems: What Breaks, Where Originates, and How to Fix
Liangyu Chen, Youpeng Ma, Yulong Ye et al. (+2 more)
Retrieval-Augmented Generation (RAG) has emerged as a dominant paradigm for improving Large
Language Model (LLM) by intertwining it with other components, such as a vector/graph database. Therefore, the resulting RAG systems are often highly configurable, under which a non-trivial amount of failures are related to the configurations. However, there remains a lack of systematic understanding regarding the configuration issues for RAG systems. To bridge this gap, in this paper, we conduct the first large-scale empirical study of 654 real-world configuration issues collected from 10 diverse RAG systems. Drawing on this, we provide a complete categorization of the symptoms, root causes, relevant stages, and fix patterns of configuration issues for RAG systems, as well as several new observations. Our findings not only reveal the characteristics of configuration issues, but also strongly link them to the unique pipeline stages of RAG, e.g., we found that most symptoms can be observed at the early or late stage, while the stage of the root cause might not be at the same stage as where the symptom is identified. We also provide nine actionable insights to the practitioners on testing, localizing, and fixing the configuration issues for RAG systems.
Read the full abstract ▾
All authors Liangyu Chen, Youpeng Ma, Yulong Ye, Zezhen Xiang, Tao Chen
Artificial Intelligence
Rethinking Quality in Large Language Model Routing: Security-Conditioned Model-and-Budget Selection
Yichi Zhang, Zhenxiang Huang, Yuyang Zhang et al. (+1 more)
Large
language model (LLM) routers reduce serving cost by choosing which model to call and how many output
tokens to grant it for each query. Existing routers rank model-budget actions using a single quality metric. This does not work for mixed workloads. For ordinary queries, a good answer is correct and useful (utility); for security-sensitive queries, a good answer refuses harmful requests or resists malicious instructions (security). These two objectives prefer different models and budgets. We propose SCRouter, a router that learns when to prioritize security and how to allocate the
token budget. For each model-budget pair, it estimates the probability that the response finishes within budget. It uses this probability to predict utility and security separately. A small probe determines how to weight these two predictions for each query. SCRouter then selects the model and budget by balancing the weighted utility and security against cost. We also build SCRouterBench, a
benchmark of 38,869 utility and security queries spanning nine security dimensions, each answered under all 144 combinations of 12 models and 12 budgets, plus 2,753 out-of-domain queries. SCRouter improves the baselines by 13-29 points in-domain and 19-23 points out-of-domain.
Read the full abstract ▾
All authors Yichi Zhang, Zhenxiang Huang, Yuyang Zhang, Yuchen Yang
Artificial Intelligence
Input-Conditioned Search for Efficient Test-Time Scaling of Recursive Reasoners
Haochen Shi, Tianwen Fu, Qianggang Ding et al. (+7 more)
People often tackle difficult problems by thinking more deeply or exploring more possibilities. Similarly, recursive reasoners, which reason by repeatedly applying a learned update rule to latent states, can also scale inference through more updates (depth) or additional trajectories generated by perturbing latent states (width). When people hit a dead end in their thinking, they may step back and approach the problem from a fresh angle. However, neither depth scaling nor perturbation-based width scaling of recursive reasoners can reliably achieve this: both may converge to incorrect attractors. To address this, we propose to explore diverse
reasoning traces by applying different update maps to the same latent state, without any additional training. We realize this idea through Input-Conditioned Search (ICS), which searches over input conditions to induce alternative update maps from the pretrained reasoner. Across six
reasoning tasks, ICS achieves the highest mean accuracy, while requiring up to two orders of magnitude fewer model FLOPs than the evaluated scaling baselines. Controlled experiments confirm that ICS can redirect trajectories from the same state. Our theoretical analysis and rigorous numerical verification also confirm that ICS can locally eliminate incorrect attracting fixed points. Together, these findings establish ICS as an effective axis for efficient test-time scaling and motivate further research on shaping recursive
reasoning during training and at test time.
Read the full abstract ▾
All authors Haochen Shi, Tianwen Fu, Qianggang Ding, Yinhan Lu, Yizhan Li, Huan Zhang, Yajie Zhao, Xingdi Yuan, Hengyang Lu, Bang Liu
Artificial Intelligence · Cross-listed from cs.MA
On Symmetry for Characterizing Agent Heterogeneity in Multi-Agent Systems
Jianhong Wang and Jonathan Lawry
Understanding
agent heterogeneity can inform the design of shared and
agent-specific policy representations, however existing criteria provide no unified structural test. We study finite-horizon POSGs and define a type-compatible transformation as a policy-evaluation
symmetry when complete policy evaluation commutes with it for every decentralized behavioral policy. Policy-evaluation
symmetries form a group for which
agent orbits define structural
agent types. This classification does not depend on the particular policy chosen or the distribution of initial states. We prove that every componentwise environment automorphism is a policy-evaluation
symmetry, but that the converse does not hold. Applying the framework to cooperative Dec-POMDPs shows that
agents can belong to different structural types even when they share a common reward. For prescribed weighted reward aggregation, we derive an exact value identity and sufficient conditions for the resulting Dec-POMDP to inherit the policy-evaluation
symmetries of the source POSG. Finally, we give sufficient conditions under which repeated model factorizations induce policy-evaluation
symmetry, and show that preserving an optimal Boltzmann policy set neither implies nor follows from this
symmetry in general.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
APTree: Extended Behavior Trees for Adaptive and Scalable Automated Robotic Task Planning in Construction
Shermin Sherkat, Thomas Wortmann and Andreas Wortmann
Task planning for
robots in off-site construction prefabrication is primarily manual, leading to planning inefficiencies and reluctance to adopt robotic solutions. Existing approaches provide limited support for scalable, modular automated task planning and localized runtime replanning. This paper proposes APTree, a model-driven framework that integrates automated task planning with modular task execution using behavior trees as the execution model. APTree repeatedly applies and coordinates heterogeneous symbolic planners to automatically generate and compose modular task plans at runtime. To support this approach, we develop a domain-specific
language for the formal specification and validation of task plans, together with a compatible execution engine. We evaluate APTree on the real-world livMatS Biomimetic Shell pavilion, where it models the construction robotic fabrication domain, detects injected specification and planning errors at design time and runtime, and automatically generates large-scale task plans by coordinating three symbolic planners across numerous planning modules. A machine-checked formal analysis further shows that composed execution and local replanning are safe under explicitly stated assumptions, including planner soundness and non-interference of generated plans, providing practitioners with explicit conditions for dependable plan composition. APTree thus enables construction practitioners to leverage automated task planning without requiring detailed knowledge of symbolic planning techniques or APTree's internal implementation.
Read the full abstract ▾
Artificial Intelligence
Lost in Implicit Routing: How LLM Agents Fail to Load Relevant Skills at Scale
Jinlin Liu, Wei Chen and Xiaojin Zhang
Reusable skills let LLM
agents load task-specific instructions on demand, but they make execution depend on implicit routing over skill metadata. We test whether this design scales by holding SkillsBench tasks and their golden skills fixed while adding irrelevant skills. We find skill routing collapse:
agents increasingly load fewer or no skills, even though the relevant skills remain available. With 1,000 irrelevant skills, loaded-skill recall falls from 73.6% to 27.4% and the zero-load rate rises from 9.3% to 51.6%, whereas explicit selection from the same library retains 73.9% recall. Loading other skills, including plausible alternatives to golden skills, is too rare to explain this decline. Masking only the descriptions of irrelevant skills largely restores skill loading, yet with the metadata intact, explicitly asking
agents to plan skill use still yields lower recall than preselection. These findings point to semantic interference from descriptions and instruction interference when skill choice competes with task execution. We also evaluate explicit preselection as a practical mitigation, including its coverage and latency trade-offs. Our analysis informs skill protocols and evaluation for
agents with growing skill libraries.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
The Geometry of Language Model Logits: Generic Structure and Learned Departures
Swathi Shree Narashiman, Aravind Ramana Venkatramanan and Nambi Seshadri
Autoregressive
language models produce, at every prediction step, a vocabulary-sized logit vector whose structure determines the conditional next-
token distribution. We empirically identify that the ordered vocabulary-head logits follow a remarkably consistent log-linear relation \[ z_{(r)} \approx \mu - \alpha \log r. \] across prediction contexts, architectures, and scales, emerging progressively during pretraining. To our knowledge, this is the first systematic analysis of the logit geometry underlying the distributional assumptions used in the prior decoding work. While controlled order-statistic baselines spanning bounded, light-tailed, and heavier-tailed distributions can largely explain the observed log-linear geometry, they do not explain the systematic shift in the slope $\alpha$: trained
language models exhibit $\alpha \approx 1.4$–$1.7$, sitting well above the initialization baseline ($\alpha \approx 0.2$) . Moreover, $\alpha$ correlates with predictive performance, suggesting that it captures learned structure beyond generic extreme-value effects.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CY
A Global View of How LLMs Value Human Time
Yifan He, Manon Reusens, Sofie Goethals et al. (+1 more)
Large
language models (LLMs) increasingly act as personal assistants; however, little is known about how they capture national-level preferences on a global scale. Given that "time" is a universal and representative concept, we examine LLMs' time preferences by varying only users' country and measuring the compensation LLMs deem appropriate for a 60-minute appointment delay. We then compare LLMs' responses with GDP per capita on Purchasing Power Parity (GDP_PPP), Mean Income, and a literature-based index (Patience Score). Our results across 193 countries and 5 LLMs demonstrate that people's time across the world is evaluated in noticeably different ways among LLMs, as well as varying levels of robustness under
prompt variations. LLMs consistently recommend the highest compensation for Europe, but mostly the lowest for Africa. Based on GDP_PPP, LLMs underestimate the value of time for users living in Asia and South America. These findings indicate that geographical patterns are embedded in LLMs and may translate in different treatment in agentic settings.
Read the full abstract ▾
All authors Yifan He, Manon Reusens, Sofie Goethals, David Martens
Artificial Intelligence
The Smart Buildings Challenge: Learning Foundation Models for Real-World Optimization at Scale
Judah Goldfeder, Ayush Rai, Philippe M. Wyder et al. (+11 more)
Buildings cause nearly 40% of carbon emissions in the US. Of total building emissions, HVAC systems are responsible for approximately 40-60% and account for 15% of global energy consumption. Despite advances in artificial intelligence, most buildings still operate as inefficiently as they did decades ago. Building optimization, perhaps more so than any other optimization problem, has enormous potential for Machine Learning (ML) driven impact on
climate change, in both the short and long term. One of the biggest obstacles for building optimization is the lack of policy transfer. Most building control systems, whether simulation or learning based, don't generalize: they only work well for the building they were developed for, but not for other buildings. To solve this problem, we propose a challenge for the development of building foundation models that easily adapt to new buildings with minimal configuration and thereby enable building optimization to scale across the globe. Participants will build control policies on a set of simulated buildings, and have their policies evaluated on an unseen set of buildings. The buildings vary in size, layout, weather, material properties, and HVAC system. Successful outcomes from this initiative promise a massive leap forward in global sustainability efforts and demonstrate the great potential of ML to combat
climate change.
Read the full abstract ▾
All authors Judah Goldfeder, Ayush Rai, Philippe M. Wyder, Na Li, Victoria Dean, Donna Vakalis, Jiong Lin, Bing Dong, Michael Rossetti, Nathan Kutz, Sohei Arisaka, John Sipple, Hod Lipson, Yoshua Bengio
Artificial Intelligence · Cross-listed from cs.CV
EgoOmniWorld: Unifying Egocentric Understanding via Ego-Action-World Interactions
Songruo Chen, Yuqian Fu, Yu Li et al. (+7 more)
Egocentric
video understanding spans diverse capabilities, modalities, and evaluation formats, yet these are often studied in isolation despite their tight coupling in real-world interaction. We propose the Ego-Action-World interaction loop as a unifying principle: the wearer perceives the world, forms goals, acts, changes the world, and uses feedback to guide subsequent behavior. Based on this view, we introduce EgoOmniWorld, a
benchmark organized around four complementary interaction structures: Node captures ego, action, and world states; Edge captures directed relations among them; Chain captures linked dependencies; and Loop captures feedback-closed interactions. We develop an interaction-first, evidence-grounded data pipeline that extracts structured interactions from egocentric
videos, grounds them in visual and auditory evidence, and generates questions according to their interaction structure. Required modalities and single- or multi-turn formats follow naturally from the supporting evidence and dependencies. EgoOmniWorld contains 24,166 QA groups, including 20,284 training and 3,882 test samples. Experiments on representative open- and closed-source
multimodal large
language models reveal substantial and uneven limitations across interaction structures. Models trained on EgoOmniWorld improve in-domain interaction understanding and generalize to existing egocentric
benchmarks. EgoOmniWorld provides a unified foundation for evaluating and advancing egocentric
video understanding. Data, code, and models will be released.
Read the full abstract ▾
All authors Songruo Chen, Yuqian Fu, Yu Li, Mingkang Dong, Sifan Zhou, Bin Ren, Xu Zheng, Tianwen Qian, Mohamed Elhoseiny, Yanwei Fu
Artificial Intelligence · Cross-listed from cs.CL
Aligning Faithfulness for Theory-Scale Autoformalization
Yuming Feng, Tadd Mao, Dhruva Arekar et al. (+4 more)
Autoformalization (AF) requires more than compilation: formal statements must preserve the meaning of their informal sources, a property known as faithfulness. An emerging frontier is the automated evaluation of this property, yet most existing methods operate at the statement level and produce whole-item verdicts, failing to reveal how individual expressions correspond or which contextual interpretations support the judgment. This omission is consequential because AF is inherently theory-scale: target statements derive their meaning from surrounding mathematical material and unstated conventions. We formulate faithfulness evaluation as a context-aware align-then-label problem: identifying phrase-level correspondences between informal and formal statements and assigning a faithfulness label to each under explicitly specified contextual interpretations and assumptions. We also introduce FaithAlign, the first theory-scale faithfulness
benchmark comprising 460 items and 4,982 annotated phrase-level links, alongside FaithAlign-Gap, a companion set of 146 items with informal context and assumptions withheld. Each FaithAlign item includes the target statement pair, relevant context, formal dependencies, and, where needed, assumptions specifying one given interpretation. Across 8 models, GPT-6 Astra achieves the highest defect F1 at 47.8% and identifies all defects in only 4.5% of items with two or more defects. On FaithAlign-Gap, Astra identifies only 28.2% of undecidable items as such. These results show the difficulty of recovering context-dependent correspondences, establishing FaithAlign as a testbed for fine-grained, context-aware faithfulness evaluation.
Read the full abstract ▾
All authors Yuming Feng, Tadd Mao, Dhruva Arekar, Tianjun Zhong, Jiani Huang, Xujie Si, Ziyang Li
Artificial Intelligence
Judges Follow the Reference: The Derivation Gap in Verifying Tool-Call Arguments
Linhai Ma, Yan Wang, Lingfei Qian et al. (+2 more)
Tool-call judges evaluate argument values proposed by LLM
agents. We find that they accept a correct value less often when their input holds no reference for it, that is, no text that says what the value should be. This happens when the value is a default value from the function schema that the user did not mention, when it has to be inferred, or when the text it comes from was cut from the judge input. We call this drop in acceptance the derivation gap. It appears across all seven judges we evaluate, including System One models CLM and Jev. The gap is not only a truncation artifact: correct default values have the largest gap on a
benchmark with no truncation. A controlled source-relocation test shows that judges follow the reference in their input: they favor a visible incorrect value over a correct value whose source is hidden. We introduce DeRef (Derived Reference): a separate model reads the full history without seeing the candidate and writes a reference for the judge. Across three
benchmarks, DeRef improves overall argument AUROC for all judges tested. On the controlled test, it raises AUROC from chance or below to between 98.40 and 99.59.
Read the full abstract ▾
All authors Linhai Ma, Yan Wang, Lingfei Qian, Yupeng Cao, Victor Gutierrez Basulto
Artificial Intelligence · Cross-listed from eess.SP
Task-evoked eye movements as candidate digital biomarkers of depression severity in adolescents and young adults
Youxiang Sun, Yujie Nie, Zhuoru Wu et al. (+6 more)
Objective digital measures of depression severity could complement conventional symptom assessments and facilitate more timely clinical evaluation. Here, we investigated whether task-evoked eye movements provide interpretable behavioral markers of depression severity in adolescents and young adults. In this single-center, cross-sectional study, 155 participants aged 14–20 years completed emotional-spatial free-viewing and
memory-guided recall tasks and were stratified by 17-item Hamilton Depression Rating Scale scores. We examined 19 prespecified eye-movement features across six task-informed domains using
covariate-adjusted ordinal-trend models. Greater depression severity was associated with lower gaze-allocation
entropy (standardized $\beta=-0.599$), fewer inter-area switches ($\beta=-0.526$), longer first target-entry latency ($\beta=0.572$), and lower target dwell proportion ($\beta=-0.678$; all false-discovery-rate-adjusted $q<0.001$). Emotion-specific attentional-bias measures were not associated with severity, whereas associations involving fixation dispersion weakened after adjustment for anxiety symptoms. In a complementary machine-learning analysis incorporating 44 candidate eye-movement features and demographic
covariates, a three-stage cascade classifier achieved the highest four-class stratification performance (accuracy, 0.665; macro-F1, 0.670; area under the receiver operating characteristic curve, 0.869), outperforming single-stage and two-stage approaches. Together, these findings identify altered visual exploration and
memory-guided gaze control as candidate digital biomarkers of depression severity, rather than generalized negative attentional bias. These results support further prospective evaluation of task-evoked eye tracking as a complementary approach for timely severity assessment and clinical decision support.
Read the full abstract ▾
All authors Youxiang Sun, Yujie Nie, Zhuoru Wu, Mengyuan Yu, Wenkong Wang, Xiang Li, Yumei Wang, Xin Ma, Lizhou Fan
Artificial Intelligence
ReTerm: Recovery Training for Self-Improving Terminal Agents
Wei Xie, Kaituo Feng, Shiming Fu et al. (+3 more)
Terminal
agents have shown strong abilities in complex coding tasks. However, they often struggle to recover from errors during long-running execution (e.g., command errors, faulty implementations), eventually leading to task failures. To this end, we present ReTerm, a self-improving terminal
agent trained by learning how to recover from its own failed terminal tasks. To support its training, we construct a customized data pipeline and produce ReTerm-10K, a high-quality
dataset that provides effective recovery demonstrations for SFT cold start. Furthermore, we propose ReTerm-GRPO, an online RL
algorithm that forks from the
agent's own failure states to generate multiple independent self-recovery attempts. By learning from whether these attempts successfully complete the task, the
agent self-improves through its own recovery experience. Built on Qwen3.5-9B, our ReTerm-9B achieves 34.8% Pass@1 on Terminal-Bench 2.0, improving over the base model by 14.6 points. Further analysis shows that our method achieves consistently higher recovery rates across diverse error categories compared with standard GRPO training. All code, data and model are released at https://github.com/twothsnd/ReTerm.
Read the full abstract ▾
All authors Wei Xie, Kaituo Feng, Shiming Fu, Yong Liu, Xiangyu Yue, Shuangjia Zheng
Artificial Intelligence · Cross-listed from cs.RO
MemoryVLN: From Ego-Aware Memory Representation to Cross-Context Memory Reuse in Vision-Language Navigation
Xueyao Wei, Yifei Wang, Delun Li et al. (+4 more)
Large
vision-
language models (LVLMs) have shown strong potential for embodied
vision-
language navigation (VLN), yet their performance degrades on long-horizon trajectories due to context-length limits, redundant historical observations, and unstable spatial
alignment across time. These issues make it difficult to both compress long navigation histories and reliably reuse past spatial experience. To address these limitations, we propose MemoryVLN, a unified
memory-driven navigation system built around
memory construction and management. During
memory construction, Long Sequence Compression employs forgetting-curve-inspired temporal sampling to preserve dense recent evidence and sparse yet representative distant observations under a limited
token budget. Ego-Aware
Memory Representation further integrates 2D semantic
tokens, 3D
geometric tokens, and 6D
agent poses, allowing each
memory to encode both what was observed and where it was observed. During
memory management, Hierarchical
Memory Bank maintains in-episode and context-level
memories through cross-context warm-start and confidence-controlled write-back.
Memory Enhancement then retrieves the Top-$K$ relevant
memories according to the current navigation state and fuses them with the current visual
tokens. Together, these mechanisms transform raw navigation history into compact, spatially grounded, selectively retrievable, and reusable navigation experience. Extensive experiments on R2R-CE and RxR-CE demonstrate consistent improvements over the state-of-the-art baselines by a substantial margin (i.e., 2% and 6.5% in successful rate).
Read the full abstract ▾
All authors Xueyao Wei, Yifei Wang, Delun Li, Shaoan Wang, Junzhi Yu, Wenjie Song, Xingyu Chen
Artificial Intelligence · Cross-listed from cs.LG
Readout Scaling Shapes Sparse Feature Recovery
Brian Xiao and Sarah Wilson
Readout scaling changes what a sparse autoencoder (SAE) learns, but the benefit of removing it depends on the data. We use readout-canonicalized decision-space (RC-DS) coordinates to remove singular-value gains while preserving retained decision information. At a fixed readout condition number, changing the generating distribution reverses which training coordinates recover known factors best. In our controlled coordinate-matching study, recovery is highest when training restores the space in which the factors were generated. When factors are generated before readout scaling, RC-DS increases mean sparse-factor recovery from 0.506 to 0.913 at condition number 100, compared with centered-logit training under the same budget. An exact wrapper reproduces RC-DS training to numerical precision, isolating training geometry rather than additional information. In neural policies, RC-DS improves mean logit reconstruction under the tested budgets. On three fixed
language-model checkpoints, RC-DS has higher mean gain-free reconstruction in all 18 tested task-sparsity settings, while centered-logit SAEs reconstruct output logits better in 17. Small-label-head controls show that longer training reduces the reconstruction gap from 0.0198 to 0.0016. Together, these results show how readout scaling, the data distribution, and training shape sparse learning. RC-DS provides an information-preserving method for studying and controlling the readout's contribution. Separate label-detection and steering tests distinguish these reconstruction effects from task-specific feature usefulness.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Learning Perturbation-Specific Representations through Conditional Energy Prior for Cellular Response Prediction
Yuchen Yan and Yatao Bian
AI-based cellular perturbation prediction is essential to accelerate drug discovery and therapeutic identification. However, perturbation-specific signals are generally obscured by cellular complexity, making it difficult for existing methods to correctly capture perturbation effects. To address this problem, we propose PerturbEnergy, which introduces a conditional energy-based prior over perturbation representations. It assigns lower energy (higher probability) to representations compatible with the given perturbation and higher energy to others, thereby explicitly tilting the latent space toward perturbation-preferred regions. Besides, PerturbEnergy can be integrated into autoencoder architectures as we find its Evidence Lower Bound (ELBO) is equivalent to the ELBO of the complete autoencoder. We further prove that optimizing PerturbEnergy increases the lower bound on the conditional mutual information between representation and perturbation. Extensive experiments on four
benchmark datasets demonstrate the effectiveness of our PerturbEnergy: it improves perturbation recognition especially in latent space with improvement of up to 83.2%; It also achieves state-of-the-art performance for predicting top-50/100/200 differentially expressed gene(DEG) response and remaining competitive on highly variable gene(HVG) prediction. The code is provided: https://github.com/BlueWhaleLab/PerturbEnergy
Read the full abstract ▾
Artificial Intelligence
Autonomous Science Bench: Evaluating AI Agents for Autonomous Scientific Discovery
Yibo Wen and Han Liu
AI
agents for autonomous scientific discovery must decide which experiments to run under a limited budget. Most
benchmarks for science
agents provide fixed
datasets, while others that support active experimentation largely rely on simulation. We introduce Autonomous Science Bench, an initial release comprising six budgeted design-build-test-learn campaigns grounded in published experiments across biology, chemistry, and materials science.
Agents select batches of designs, receive measurements from a lab service, and use the results to guide subsequent experiments before delivering a predictive model. Success requires passing two calibrated gates: a discovery gate, which evaluates the designs the
agent measured, and a learning gate, which evaluates its model's ranking of held-out designs. In a pilot evaluation of eight
agent-model configurations and 144 trials, the best configuration succeeds in 67% of trials, while the hardest campaign is solved in only three of 24 attempts. The two gates reveal different weaknesses: some
agents build accurate rankings but find too few promising designs, while others find promising designs but fail to predict which unseen designs will perform well. These differences are obscured by overall success rates alone. We release version 0.1 of the campaigns, lab runtime, and public leaderboard at https://yibow.me/autonomous-science-bench.
Read the full abstract ▾
Artificial Intelligence
Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence
Thinker Team Ubtech:, Peng Gao, Jichao Jiao et al. (+8 more)
World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the functions they support. In particular, visual dynamics,
geometric structure, and executable action are commonly coupled through separate interfaces or downstream modules rather than being jointly represented and evolved within a shared dynamics model. In this work, we present Thinker-WM, a unified
multimodal world model that jointly models and generates RGB, metric depth, and
robot action through a shared intermediate Unified Dynamics
Transformer with modality-specific encoders and decoders. The model performs
multimodal unification at the representation level: appearance, geometry, and action are transformed into a common dynamic representation and jointly evolved by shared parameters, while their distinct output structures are preserved. The same parameterization supports multiple functions, including joint
video-action generation, world prediction, policy inference, action-conditioned generation,
video-to-action generation, and inverse dynamics. To examine whether the shared representation captures action-related information, we introduce a lightweight motion probe that shows the frozen visual representation contains recoverable motion information. Thinker-WM is trained on a heterogeneous corpus comprising approximately 20K hours of egocentric
video, 10K hours of real-
robot data, and 1K hours of simulation data, including approximately 4K hours of real-
robot data with aligned depth supervision. Extensive experiments demonstrate strong performance across embodied world-model evaluations. Thinker-WM ranks first on EWMBench, WorldOlympiad and WorldArena, while controlled studies further demonstrate the benefits of action conditioning,
geometric interaction, and multi-function training within the shared representation.
Read the full abstract ▾
All authors Thinker Team Ubtech:, Peng Gao, Jichao Jiao, Chen Liao, Daqin Luo, Yiming Ma, Baiyu Pan, Hailin Shi, Jiyuan Wang, Xiangming Xu, Xiongwei Zhao
Artificial Intelligence
The Singularity Has a Distribution: Exact Timing Laws and Detection Limits for a Stochastic Intelligence Explosion
Xiaoyu Li, Zheng Gao, Chiwun Yang et al. (+1 more)
Automating AI research creates a positive feedback loop between software capability and research effort. We study a
stochastic realisation of this loop in which independent, positive, multiplicative improvements arrive at a rate proportional to a power of current capability. Within this model, randomness leaves the deterministic explosion threshold unchanged but replaces the blow-up date by an explicit distribution. For equal-sized improvements the explosion time is a geometrically weighted sum of independent exponentials; its $q$-series representation gives its transforms, moments and tail behaviour, including a two-term small-time asymptotic. For Lomax-distributed improvement sizes the explosion time is exactly Gamma distributed. Self-similarity implies that, with parameters known, the relative uncertainty about the remaining time does not vanish as explosion approaches. We formulate a fixed-sample test between constant and geometrically increasing idea-arrival rates. If the baseline rate and the alternative log-rate increment $\ell$ are known, the minimum sample size at error level $\alpha$ is asymptotic to $(12z_\alpha^2)^{1/3}\ell^{-2/3}$, where $z_\alpha$ is the upper-$\alpha$ standard normal
quantile. Uncertainty about the growth exponent near its critical value produces a $1/t$ tail with an explicit constant, and hence an infinite conditional mean waiting time, although explosion is certain given a positive exponent. Separate extensions show that a fixed deployment delay prevents explosion in the specified delayed-feedback models, that power-law diversion of effort shifts the growth threshold, and that in a deterministic oversight race a fixed allocation share does not change the asymptotic regime, except at a critical equality of exponents. Numerical illustrations, which are not empirical forecasts, show how timing and detectability depend on the granularity of improvements.
Read the full abstract ▾
All authors Xiaoyu Li, Zheng Gao, Chiwun Yang, Zhizhou Sha