Robotics
Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigation
Zhenxing Xu, Yihong Lu, Weidong Bao et al. (+6 more)
Current Visual-
Language Navigation (VLN) methodologies face a trade-off between semantic understanding and control precision. While
Multimodal Large
Language Models (MLLMs) offer superior
reasoning, deploying them as low-level controllers leads to high latency, trajectory oscillations, and poor generalization due to weak
geometric grounding. To address these limitations, we propose Fly0, a framework that decouples semantic
reasoning from
geometric planning. The proposed method operates through a three-stage pipeline: (1) an MLLM-driven module for grounding natural
language instructions into 2D pixel coordinates; (2) a
geometric projection module that utilizes depth data to localize targets in 3D space; and (3) a
geometric planner that generates collision-free trajectories. This mechanism enables robust
navigation even when visual contact is lost. By eliminating the need for continuous inference, Fly0 reduces computational overhead and improves system stability. Extensive experiments in simulation and real-world environments demonstrate that Fly0 outperforms state-of-the-art baselines, improving the Success Rate by over 20% and reducing
Navigation Error (NE) by approximately 50% in unstructured environments. Our code is available at https://github.com/xuzhenxing1/Fly0.
Read the full abstract ▾
All authors Zhenxing Xu, Yihong Lu, Weidong Bao, Zhengqiu Zhu, Jingxuan Zhou, Zhichuang Wang, Ji Wang, Lihua Liu, Wei He
Robotics
DORA: Dataflow Oriented Robotic Architecture
Xiaodong Zhang, Baorui Lv, Xavier Tao et al. (+5 more)
Robotic middleware serves as the foundational infrastructure, enabling complex robotic systems to operate in a coordinated and modular manner. In data-intensive robotic applications, especially in industrial scenarios, communication efficiency directly impact system responsiveness, stability, and overall productivity. However, existing robotic middleware exhibit several limitations: (1) they rely heavily on (de)serialization mechanisms, introducing significant overhead for large-sized data; (2) they lack efficient and flexible support for heterogeneous data sizes, particularly in intra-
robot communication and Python-based execution environments. To address these challenges, we propose Dataflow-Oriented Robotic Architecture (DORA) that enables explicit data dependency specification and efficient zero-copy data transmission. We implement the proposed framework as an open-source system and evaluate it through extensive experiments in both simulation and real-world robotic environments. Experimental results demonstrate substantial reductions in latency and CPU overhead compared to state-of-the-art middleware.
Read the full abstract ▾
All authors Xiaodong Zhang, Baorui Lv, Xavier Tao, Xiong Wang, Jie Bao, Yong He, Yue Chen, Zijiang Yang
Robotics · Cross-listed from cs.AI
X-Blocks: Linguistic Building Blocks of Natural Language Explanations for Automated Vehicles
Ashkan Y. Zadeh, Xiaomeng Li, Andry Rakotonirainy et al. (+3 more)
Natural
language explanations play a critical role in establishing trust and acceptance of automated vehicles (AVs), yet existing approaches lack systematic frameworks for analysing how humans linguistically construct driving rationales across diverse scenarios. This paper introduces X-Blocks (eXplanation Blocks), a hierarchical analytical framework that identifies the linguistic building blocks of natural
language explanations for AVs at three levels: context, syntax, and lexicon. At the context level, we propose RACE (
Reasoning-Aligned Classification of Explanations), a multi-LLM ensemble framework that combines Chain-of-Thought
reasoning with self-consistency mechanisms to robustly classify explanations into 32 scenario-aware categories. Applied to human-authored explanations from the Berkeley DeepDrive-X
dataset, RACE achieves 91.45 percent accuracy and a Cohens kappa of 0.91 against cases with human annotator agreement, indicating near-human reliability for context classification. At the lexical level, log-odds analysis with informative Dirichlet priors reveals context-specific vocabulary patterns that distinguish driving scenarios. At the syntactic level, dependency parsing and template extraction show that explanations draw from a limited repertoire of reusable grammar families, with systematic variation in predicate types and causal constructions across contexts. The X-Blocks framework is
dataset-agnostic and task-independent, offering broad applicability to other automated driving
datasets and safety-critical domains. Overall, our findings provide evidence-based linguistic design principles for generating scenario-aware explanations that support transparency, user trust, and cognitive accessibility in automated driving systems.
Read the full abstract ▾
All authors Ashkan Y. Zadeh, Xiaomeng Li, Andry Rakotonirainy, Ronald Schroeter, Sebastien Glaser, Zishuo Zhu
Robotics
A Distributed Multi-Modal Sensing Approach for Human Activity Recognition in Real-Time Human-Robot Collaboration
Valerio Belcamino, Nhat Minh Dinh Le, Quan Khanh Luu et al. (+3 more)
Human activity recognition (HAR) is fundamental in human-
robot collaboration (HRC), enabling
robots to respond to and dynamically adapt to human intentions. This paper introduces a HAR system combining a modular data glove equipped with Inertial Measurement Units and a
vision-based tactile sensor to capture hand activities in contact with a
robot. We tested our activity recognition approach under different conditions, including offline classification of segmented sequences, real-time classification under static conditions, and a realistic HRC scenario. The experimental results show a high accuracy for all the tasks, suggesting that multiple collaborative settings could benefit from this multi-modal approach.
Read the full abstract ▾
All authors Valerio Belcamino, Nhat Minh Dinh Le, Quan Khanh Luu, Alessandro Carfì, Van Anh Ho, Fulvio Mastrogiovanni
Robotics
Modular Isoperimetric Soft Robotic Truss for Lunar Applications
Mihai Stanciu, Isaac Weaver, Adam Rose et al. (+5 more)
We introduce a large-scale robotic system designed as a lightweight, modular, and reconfigurable structure for lunar applications. The system consists of truss-like robotic triangles formed by continuous inflated fabric tubes routed through two robotic roller units and a connecting unit. A newly developed spherical joint enables up to three triangles to connect at a vertex, allowing construction of truss assemblies beyond a single octahedron. When deflated, the triangles compact to approximately the volume of the roller units, achieving a stowed-to-deployed volume ratio of 1:18.3. Upon inflation, the roller units pinch the tubes, locally reducing bending stiffness to form effective joints. Electric motors then translate the roller units along the tube, shifting the pinch point by lengthening one edge while shortening another at the same rate, thereby preserving a constant perimeter (isoperimetric). This shape-changing process requires no additional compressed air, enabling untethered operation after initial inflation. We demonstrate the system as a 12-degree-of-freedom
solar array capable of tilting up to 60 degrees and sweeping 360 degrees, and as a 14-degree-of-freedom locomotion device using a step-and-slide gait. This modular, shape-adaptive system addresses key challenges for sustainable lunar operations and future space missions.
Read the full abstract ▾
All authors Mihai Stanciu, Isaac Weaver, Adam Rose, James Wade, Kaden Paxton, Chris Paul, Spencer Stowell, Nathan Usevitch
Robotics
Moving On, Even When You're Broken: Fail-Active Trajectory Generation via Diffusion Policies Conditioned on Embodiment and Task
Gilberto G. Briscoe-Martinez, Yaashia Gautam, Rahul Shetty et al. (+3 more)
Robot failure is detrimental and disruptive, often requiring human intervention to recover. Our
vision is 'fail-active' operation, allowing
robots to safely complete their tasks even when damaged. Focusing on 'actuation failures', we introduce DEFT, a
diffusion-based trajectory generator conditioned on the
robot's current embodiment and task constraints. DEFT generalizes across failure types, supports constrained and unconstrained motions, and enables task completion under arbitrary failure. We evaluate DEFT in both simulation and real-world scenarios using a 7-DoF robotic arm. DEFT outperforms its baselines over thousands of failure conditions, achieving a 99.5% success rate for unconstrained motions versus RRT's 42.4%, and 46.4% for constrained motions versus differential IK's 30.9%. Furthermore, DEFT demonstrates robust zero-shot generalization by maintaining performance on failure conditions unseen during training. Finally, we perform real-world evaluations on two multi-step tasks, drawer manipulation and whiteboard erasing. These experiments demonstrate DEFT succeeding on tasks where classical methods fail. Our results show that DEFT achieves fail-active manipulation across arbitrary failure configurations and real-world deployments.
Read the full abstract ▾
All authors Gilberto G. Briscoe-Martinez, Yaashia Gautam, Rahul Shetty, Anuj Pasricha, Marco M. Nicotra, Alessandro Roncone
Robotics
Accelerating Structured Chain-of-Thought in Autonomous Vehicles
Yi Gu, Yan Wang, Yuxiao Chen et al. (+8 more)
Chain-of-Thought (CoT)
reasoning enhances the decision-making capabilities of
vision-
language-action models in autonomous driving, but its autoregressive nature introduces significant inference latency, making it impractical for real-time applications. To address this, we introduce FastDriveCoT, a novel parallel decoding method that accelerates template-structured CoT. Our approach decomposes the
reasoning process into a dependency graph of distinct sub-tasks, such as identifying critical objects and summarizing traffic rules, some of which can be generated in parallel. By generating multiple independent
reasoning steps concurrently within a single forward pass, we significantly reduce the number of sequential computations. Experiments demonstrate a 3-4$\times$ speedup in CoT generation and a substantial reduction in end-to-end latency across various model architectures, all while preserving the original downstream task improvements brought by incorporating CoT
reasoning.
Read the full abstract ▾
All authors Yi Gu, Yan Wang, Yuxiao Chen, Yurong You, Wenjie Luo, Yue Wang, Wenhao Ding, Boyi Li, Heng Yang, Boris Ivanovic, Marco Pavone
Robotics
IMAGINE: Intelligent Multi-Agent Godot-based Indoor Networked Exploration
Tiago Leite, Maria Conceição and António Grilo
The exploration of unknown, Global
Navigation Satellite System (GNSS) denied environments by an autonomous communication-aware and collaborative group of Unmanned Aerial Vehicles (
UAVs) presents significant challenges in coordination, perception, and decentralized decision-making. This paper implements Multi-
Agent Reinforcement Learning (MARL) to address these challenges in a 2D indoor environment, using high-fidelity game-engine simulations (Godot) and continuous action spaces. Policy training aims to achieve emergent collaborative behaviours and decision-making under uncertainty using Network-Distributed Partially Observable Markov Decision Processes (ND-POMDPs). Each
UAV is equipped with a Light Detection and Ranging (
LiDAR) sensor and can share data (sensor measurements and a local occupancy map) with neighbouring
agents. Inter-
agent communication constraints include limited range, bandwidth and latency. Extensive ablation studies evaluated MARL training paradigms, reward function, communication system, neural network (NN) architecture,
memory mechanisms, and POMDP formulations. This work jointly addresses several key limitations in prior research, namely reliance on discrete actions, single-
agent or centralized formulations, assumptions of a priori knowledge and permanent connectivity, inability to handle dynamic obstacles, short planning horizons and architectural complexity in Recurrent NNs/
Transformers. Results show that the scalable training paradigm, combined with a simplified architecture, enables rapid autonomous exploration of an indoor area. The implementation of Curriculum-Learning (five increasingly complex levels) also enabled faster, more robust training. This combination of high-fidelity simulation, MARL formulation, and computational efficiency establishes a strong foundation for deploying learned cooperative strategies in physical robotic systems.
Read the full abstract ▾
Robotics
Latent Perspective-Taking via a Schrödinger Bridge in Influence-Augmented Local Models
Kevin Alcedo, Pedro U. Lima and Rachid Alami
Operating in environments alongside humans requires
robots to make decisions under uncertainty. In addition to exogenous dynamics, they must reason over others' hidden mental-models and mental-states. While Interactive POMDPs and
Bayesian Theory of Mind formulations are principled, exact nested-belief inference is intractable, and hand-specified models are brittle in open-world settings. We address both by learning structured mental-models and an
estimator of others' mental-states. Building on the Influence-Based Abstraction, we instantiate an Influence-Augmented Local Model to decompose socially-aware
robot tasks into local dynamics, social influences, and exogenous factors. We propose (a) a neuro-symbolic world model instantiating a factored, discrete Dynamic
Bayesian Network, and (b) a perspective-shift operator modeled as an amortized Schrödinger Bridge over the learned local dynamics that transports factored
egocentric beliefs into other-centric beliefs. We show that this architecture enables
agents to synthesize socially-aware policies in model-based reinforcement learning, via decision-time mental-state planning (a Schrödinger Bridge in belief space), with preliminary results in a MiniGrid social
navigation task.
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
Causal Flow Q-Learning for Robust Offline Reinforcement Learning
Mingxuan Li, Junzhe Zhang and Elias Bareinboim
Expressive policies based on flow-matching have been successfully applied in reinforcement learning (RL) more recently due to their ability to model complex action distributions from offline data. These
algorithms build on standard policy gradients, which assume that there is no unmeasured confounding in the data. However, this condition does not necessarily hold for pixel-based demonstrations when a mismatch exists between the demonstrator's and the learner's sensory capabilities, leading to implicit confounding biases in offline data. We address the challenge by investigating the problem of confounded observations in offline RL from a causal perspective. We develop a novel causal offline RL objective that optimizes policies' worst-case performance that may arise due to confounding biases. Based on this new objective, we introduce a practical implementation that learns expressive flow-matching policies from confounded demonstrations, employing a deep discriminator to assess the discrepancy between the target policy and the nominal behavioral policy. Experiments across 25 pixel-based tasks demonstrate that our proposed confounding-robust augmentation procedure achieves a success rate 120% that of confounding-unaware, state-of-the-art offline RL methods.
Read the full abstract ▾
Robotics
Kino-PAX$^+$: Near-Optimal Massively Parallel Kinodynamic Sampling-based Motion Planner
Nicolas Perrault, Qi Heng Ho and Morteza Lahijanian
Sampling-based motion planners (SBMPs) are widely used for
robot motion planning with complex kinodynamic constraints in high-dimensional spaces, yet their serial computation design results in planning speeds that scale poorly with problem complexity. Recent efforts to parallelize SBMPs have achieved significant speedups in finding feasible solutions; however, they provide no guarantees of optimizing an objective function. We introduce Kino-PAX$^{+}$, a massively parallel kinodynamic SBMP with asymptotic near-optimal guarantees. Kino-PAX$^{+}$ builds a sparse tree of dynamically feasible trajectories by decomposing traditionally serial operations into three massively parallel subroutines. The
algorithm focuses computation on the most promising nodes within local neighborhoods for propagation and refinement, enabling rapid improvement of solution cost. We prove that, while maintaining probabilistic $\delta$-robust completeness, this focus on promising nodes ensures asymptotic $\delta$-robust near-optimality. Our results show that Kino-PAX$^{+}$ finds solutions up to three orders of magnitude faster than existing serial methods and achieves lower solution costs
Read the full abstract ▾
Robotics
Language Movement Primitives: Grounding Language Models in Robot Motion
Yinlong Dai, Benjamin A. Christie, Daniel J. Evans et al. (+2 more)
Enabling
robots to perform novel manipulation tasks from natural
language instructions remains a fundamental challenge in robotics, despite significant progress in generalized problem solving with foundational models. Large
vision and
language models (VLMs) are capable of processing high-dimensional input data for visual scene and
language understanding, as well as decomposing tasks into a sequence of logical steps; however, they struggle to ground those steps in
embodied robot motion. On the other hand, robotics foundation models output action commands, but require in-domain fine-tuning or experience before they are able to perform novel tasks successfully. At its core, there still remains the fundamental challenge of connecting abstract task
reasoning with low-level motion control. To address this disconnect, we propose
Language Movement Primitives (LMPs), a framework that grounds VLM
reasoning in Dynamic Movement Primitive (DMP) parameterization. Our key insight is that DMPs provide a small number of interpretable parameters, and VLMs can set these parameters to specify diverse, continuous, and stable trajectories. Put another way: VLMs can reason over free-form natural
language task descriptions, and semantically ground their desired motions into DMPs – bridging the gap between high-level task
reasoning and low-level position and velocity control. Building on this combination of VLMs and DMPs, we formulate our LMP pipeline for zero-shot
robot manipulation that effectively completes tabletop manipulation problems by generating a sequence of DMP motions. Across 31 real-world manipulation tasks, we show that LMP achieves 65% task success as compared to 35% for the best performing baseline. See
videos at our website: https://collab.me.vt.edu/lmp
Read the full abstract ▾
All authors Yinlong Dai, Benjamin A. Christie, Daniel J. Evans, Dylan P. Losey, Simon Stepputtis
Robotics
Adaptive Linear Path Model-Based Diffusion
Yutaka Shimizu and Masayoshi Tomizuka
The interest in combining model-based control approaches with
diffusion models has been growing. Although we have seen many impressive robotic control results in difficult tasks, the performance of
diffusion models is highly sensitive to the choice of scheduling parameters, making parameter tuning one of the most critical challenges. We introduce Linear Path Model-Based
Diffusion (LP-MBD), which replaces the variance-preserving schedule with a flow-matching-inspired linear probability path. This yields a geometrically interpretable and decoupled parameterization that reduces tuning complexity and provides a stable foundation for adaptation. Building on this, we propose Adaptive LP-MBD (ALP-MBD), which leverages reinforcement learning to adjust
diffusion steps and noise levels according to task complexity and environmental conditions. Across numerical studies, Brax
benchmarks, and mobile-
robot trajectory tracking, LP-MBD simplifies scheduling while maintaining strong performance, and ALP-MBD further improves robustness, adaptability, and real-time efficiency.
Read the full abstract ▾
Robotics · Cross-listed from math.OC
Fast Near Time-Optimal Motion Planning for Holonomic Vehicles in Structured Environments
Louis Callens, Bastiaan Vandewal, Ibrahim Ibrahim et al. (+2 more)
This paper proposes a novel and efficient optimization-based method for generating near time-optimal trajectories for holonomic vehicles navigating through complex but structured environments. The approach aims to solve the problem of motion planning for planar motion systems using
magnetic levitation that can be used in assembly lines, automated laboratories or clean-rooms. In these applications, time-optimal trajectories that can be computed in real-time are required to increase productivity and allow the vehicles to be reactive if needed. The presented approach encodes the environment representation using free-space corridors and represents the motion of the vehicle through such a corridor using a motion primitive. These primitives are selected heuristically and define the trajectory with a limited number of degrees of freedom, which are determined in an optimization problem. As a result, the method achieves significantly lower computation times compared to the state-of-the-art, most notably solving a full Optimal Control Problem (OCP), OMG-tools or VP-STO without significantly compromising optimality within a fixed corridor sequence. The approach is benchmarked extensively in simulation and is validated on a real-world Beckhoff XPlanar system
Read the full abstract ▾
All authors Louis Callens, Bastiaan Vandewal, Ibrahim Ibrahim, Jan Swevers, Wilm Decré
Robotics · Cross-listed from math.OC
Sub-optimality bounds for certainty equivalent policies in partially observed systems
Berk Bozkurt, Aditya Mahajan, Ashutosh Nayyar et al. (+1 more)
In this paper, we present a generalization of the certainty equivalence principle of
stochastic control. One interpretation of the classical certainty equivalence principle for linear systems with output feedback and quadratic costs is as follows: the optimal action at each time is obtained by evaluating the optimal state-feedback policy of the
stochastic linear system at the minimum mean square error (MMSE) estimate of the state. Motivated by this interpretation, we consider certainty equivalent policies for general (non-linear) partially observed
stochastic systems that allow for any state estimate rather than restricting to MMSE estimates. In such settings, the certainty equivalent policy is not optimal. For models where the cost and the dynamics are smooth in an appropriate sense, we derive upper bounds on the sub-optimality of certainty equivalent policies. We present several examples to illustrate the results.
Read the full abstract ▾
All authors Berk Bozkurt, Aditya Mahajan, Ashutosh Nayyar, Yi Ouyang
Robotics · Cross-listed from cs.HC
Simulating Human Audiovisual Search Behavior
Hyunsung Cho, Xuejing Luo, Byungjoo Lee et al. (+2 more)
Locating a target based on auditory and visual cues$\unicode{x2013}$such as finding a car in a crowded parking lot or identifying a speaker in a virtual meeting$\unicode{x2013}$requires balancing effort, time, and accuracy under uncertainty. Existing models of audiovisual search often treat perception and action in isolation, overlooking how people adaptively coordinate movement and sensory strategies. We present Sensonaut, a computational model of
embodied audiovisual search. The core assumption is that people deploy their body and sensory systems in ways they believe will most efficiently improve their chances of locating a target, trading off time and effort under perceptual constraints. Our model formulates this as a resource-rational decision-making problem under partial observability. We validate the model against newly collected human data, showing that it reproduces both adaptive scaling of search time and effort under task complexity, occlusion, and distraction, and characteristic human errors. Our simulation of human-like resource-rational search informs the design of audiovisual interfaces that minimize search cost and cognitive load.
Read the full abstract ▾
All authors Hyunsung Cho, Xuejing Luo, Byungjoo Lee, David Lindlbauer, Antti Oulasvirta
Robotics
Bimanual High-Density EMG Control for In-Home Mobile Manipulation by Users with Quadriplegia
Jehan Yang, Eleanor Hodgson, Cindy Sun et al. (+2 more)
Mobile manipulators in the home can enable people with cervical spinal cord injury (cSCI) to perform daily physical household tasks that they could not otherwise do themselves. However, paralysis in these users often limits access to traditional
robot control interfaces such as joysticks or keyboards. In this work, we introduce and deploy the first system that enables a user with quadriplegia to control a mobile manipulator using intent from paralyzed parts of their body, using bimanual high-density electromyography (HDEMG). We develop a pair of custom, fabric-integrated HDEMG forearm sleeves, worn on both arms, that capture residual neuromotor activity from clinically paralyzed degrees of freedom and support real-time gesture-based
robot control. We achieve high classification accuracies based on motor intent across (n = 2) users with cSCI, achieving up to 98.0%. Second, by integrating
vision,
language, and motion planning modules, we introduce a shared autonomy framework that supports robust and user-driven teleoperation, with particular benefits for
navigation-intensive tasks in home environments. Finally, to demonstrate the system in the wild, we present a twelve-day in-home user study evaluating real-time use of the wearable EMG interface for daily
robot control. Together, these system components enable effective
robot control for performing activities of daily living (ADLs) and other household tasks in a real home environment.
Read the full abstract ▾
All authors Jehan Yang, Eleanor Hodgson, Cindy Sun, Zackory Erickson, Doug Weber
Robotics · Cross-listed from cs.HC
Ethical Asymmetry in Human-Robot Interaction - An Empirical Test of Sparrow's Hypothesis
Minyi Wang, Christoph Bartneck, Michael-John Turp et al. (+1 more)
The ethics of human-
robot interaction (HRI) have been discussed extensively based on three traditional frameworks: deontology, consequentialism, and virtue ethics. We conducted a mixed within/between experiment to investigate Sparrow's proposed ethical asymmetry hypothesis in human treatment of
robots. The moral permissibility of action (MPA) was manipulated as a subject grouping variable, and virtue type (prudence, justice, courage, and temperance) was controlled as a within-subjects factor. We tested moral stimuli using an online questionnaire with Perceived Moral Permissibility of Action (PMPA) and Perceived Virtue Scores (PVS) as response measures. The PVS measure was based on an adaptation of the established Questionnaire on Cardinal Virtues (QCV), while the PMPA was based on Malle et al. [39] work. We found that the MPA significantly influenced the PMPA and perceived virtue scores. The best-fitting model to describe the relationship between PMPA and PVS was cubic, which is symmetrical in nature. Our study did not confirm Sparrow's asymmetry hypothesis. The adaptation of the QCV is expected to have utility for future studies, pending additional psychometric property assessments.
Read the full abstract ▾
All authors Minyi Wang, Christoph Bartneck, Michael-John Turp, David Kaber
Robotics
PokeNet: Learning Kinematic Models of Articulated Objects from Human Observations
Anmol Gupta, Weiwei Gu, Omkar Patil et al. (+2 more)
Articulation modeling enables
robots to learn joint parameters of articulated objects for effective manipulation which can then be used downstream for skill learning or planning. Existing approaches often rely on prior knowledge about the objects, such as the number or type of joints. Some of these approaches also fail to recover occluded joints that are only revealed during interaction. Others require large numbers of multi-view images for every object, which is impractical in real-world settings. Furthermore, prior works neglect the order of manipulations, which is essential for many multi-DoF objects where one joint must be operated before another, such as a dishwasher. We introduce PokeNet, an end-to-end framework that estimates articulation models from a single human demonstration without prior object knowledge. Given a sequence of point cloud observations of a human manipulating an unknown object, PokeNet predicts joint parameters, infers manipulation order, and tracks joint states over time. PokeNet outperforms existing state-of-the-art methods, improving joint axis and state estimation accuracy by an average of over 27% across diverse objects, including novel and unseen categories. We demonstrate these gains in both simulation and real-world environments.
Read the full abstract ▾
All authors Anmol Gupta, Weiwei Gu, Omkar Patil, Jun Ki Lee, Nakul Gopalan
Robotics
AROLA: A Modular Layered Architecture for Scaled Autonomous Racing
Fam Shihata, Mohammed Abdelazim and Ahmed Hussein
Autonomous racing has advanced rapidly, particularly on scaled platforms, and software stacks must evolve accordingly. In this work, AROLA is introduced as a modular, layered software architecture in which fragmented and monolithic designs are reorganized into interchangeable layers and components connected through standardized ROS 2 interfaces. The autonomous-driving pipeline is decomposed into sensing, pre-processing, perception, localization and mapping, planning, behavior, control, and actuation, enabling rapid module replacement and objective benchmarking without reliance on custom message definitions. To support consistent performance evaluation, a Race Monitor framework is introduced as a lightweight system through which lap timing, trajectory quality, and computational load are logged in real time and standardized post-race analyses are generated. AROLA is validated in simulation and on hardware using the RoboRacer platform, including deployment at the 2025 RoboRacer IV25 competition. Together, AROLA and Race Monitor demonstrate that modularity, transparent interfaces, and systematic evaluation can accelerate development and improve reproducibility in scaled autonomous racing.
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusion
Dan Haramati, Carl Qi, Tal Daniel et al. (+3 more)
We propose a hierarchical entity-centric framework for offline Goal-Conditioned Reinforcement Learning (GCRL) that combines subgoal decomposition with factored structure to solve long-horizon tasks in domains with multiple entities. Achieving long-horizon goals in complex environments remains a core challenge in Reinforcement Learning (RL). Domains with multiple entities are particularly difficult due to their combinatorial complexity. GCRL facilitates generalization across goals and the use of subgoal structure, but struggles with high-dimensional observations and combinatorial state-spaces, especially under sparse reward. We employ a two-level hierarchy composed of a value-based GCRL
agent and a factored subgoal-generating conditional
diffusion model. The RL
agent and subgoal generator are trained independently and composed post hoc through selective subgoal generation based on the value function, making the approach modular and compatible with existing GCRL
algorithms. We introduce new variations to
benchmark tasks that highlight the challenges of multi-entity domains, and show that our method consistently boosts performance of the underlying RL
agent on image-based long-horizon tasks with sparse rewards, achieving over 150% higher success rates on the hardest task in our suite and generalizing to increasing horizons and numbers of entities. Rollout
videos are provided at: https://sites.google.com/view/hecrl
Read the full abstract ▾
All authors Dan Haramati, Carl Qi, Tal Daniel, Amy Zhang, Aviv Tamar, George Konidaris
Robotics
Flow Policy Gradients for Robot Control
Brent Yi, Hongsuk Choi, Himanshu Gaurav Singh et al. (+9 more)
Likelihood-based policy gradient methods are the dominant approach for training
robot control policies from rewards. These methods rely on differentiable action likelihoods, which constrain policy outputs to simple distributions like Gaussians. In this work, we show how flow matching policy gradients, a recent framework that bypasses likelihood computation, can be made effective for training and finetuning more expressive policies in challenging
robot control settings. We introduce an improved objective that enables success in legged locomotion,
humanoid motion tracking, and manipulation tasks, as well as robust sim-to-real transfer on two
humanoid robots. We then present ablations and analysis on training dynamics. Results show how policies can exploit the flow representation for exploration when training from scratch, as well as improved finetuning robustness over baselines.
Read the full abstract ▾
All authors Brent Yi, Hongsuk Choi, Himanshu Gaurav Singh, Xiaoyu Huang, Takara E. Truong, Carmelo Sferrazza, Yi Ma, Rocky Duan, Pieter Abbeel, Guanya Shi, Karen Liu, Angjoo Kanazawa
Robotics
HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videos
Yinhuai Wang, Qihan Zhao, Yuen Fui Lau et al. (+6 more)
Enabling
humanoid robots to perform agile and adaptive interactions has long been a core challenge in robotics. Current approaches are bottlenecked by either the scarcity of realistic interaction data or the need for meticulous, task-specific reward engineering, which limits their scalability. To narrow this gap, we present HumanX, a unified framework that compiles human
video into generalizable, real-world interaction skills for
humanoids, without per-task design. HumanX integrates two co-designed components: XGen, an anchor-centric data generation pipeline that yields physically plausible
robot interaction data from
monocular video and augments it to create a diverse, scalable training
dataset; and XMimic, an interaction-prioritized learning framework to achieve generalizable interaction skills from seconds of demonstration. Evaluated across five distinct domains–basketball, football, badminton, cargo pickup, and reactive fighting–HumanX successfully acquires 9 different skills and transfers them zero-shot to a physical Unitree G1
humanoid. The learned capabilities include complex maneuvers such as pump-fake turnaround fadeaway jumpshots without any external perception, as well as interactive tasks like sustained human-
robot passing sequences over 10 consecutive cycles–learned from a single
video demonstration. Our experiments show that HumanX achieves higher generalization success than prior methods, demonstrating a scalable pathway within a class of anchorable interactions for learning versatile, real-world
robot interactive skills.
Read the full abstract ▾
All authors Yinhuai Wang, Qihan Zhao, Yuen Fui Lau, Runyi Yu, Hok Wai Tsui, Qifeng Chen, Jingbo Wang, Jiangmiao Pang, Ping Tan