Robotics · Cross-listed from cs.LG
PriPG-RL: Privileged Planner-Guided Reinforcement Learning for Partially Observable Systems with Anytime-Feasible MPC
Mohsen Amiri, Mohsen Amiri, Ali Beikmohammadi et al. (+2 more)
This paper addresses the problem of training a reinforcement learning (RL) policy under partial observability by exploiting a privileged, anytime-feasible planner
agent available exclusively during training. We formalize this as a Partially Observable Markov Decision Process (POMDP) in which a planner
agent with access to an approximate dynamical model and privileged state information guides a learning
agent that observes only a lossy projection of the true state. To realize this framework, we introduce an anytime-feasible Model Predictive Control (MPC)
algorithm that serves as the planner
agent. For the learning
agent, we propose Planner-to-Policy Soft Actor-Critic (P2P-SAC), a method that distills the planner
agent's privileged knowledge to mitigate partial observability and thereby improve both sample efficiency and final policy performance. We support this framework with rigorous theoretical analysis. Finally, we validate our approach in simulation using NVIDIA Isaac Lab and successfully deploy it on a real-world Unitree Go2 quadruped navigating complex, obstacle-rich environments.
Read the full abstract ▾
All authors Mohsen Amiri, Mohsen Amiri, Ali Beikmohammadi, Sindri Magnuśson, Mehdi Hosseinzadeh
Robotics · Cross-listed from cs.AI
"Why This Avoidance Maneuver?" Contrastive Explanations in Human-Supervised Maritime Autonomous Navigation
Joel Jose, Andreas Madsen, Andreas Brandsæter et al. (+2 more)
Automated maritime collision avoidance will rely on human supervision for the foreseeable future. This necessitates transparency into how the system perceives a scenario and plans a maneuver. However, the causal logic behind avoidance maneuvers is often complex and difficult to convey to a navigator. This paper explores how to explain these factors in a selective, understandable manner for supervisors with a nautical background. We propose a method for generating contrastive explanations, which provide human-centric insights by comparing a system's proposed solution against relevant alternatives. To evaluate this, we developed a framework that uses visual and textual cues to highlight key objectives from a state-of-the-art collision avoidance system. An exploratory user study with four experienced marine officers suggests that contrastive explanations support the understanding of the system's objectives. However, our findings also reveal that while these explanations are highly valuable in complex multi-vessel encounters, they can increase cognitive workload, suggesting that future maritime interfaces may benefit most from demand-driven or scenario-specific explanation strategies.
Read the full abstract ▾
All authors Joel Jose, Andreas Madsen, Andreas Brandsæter, Tor A. Johansen, Erlend M. Coates
Robotics
Open-Ended Instruction Realization with LLM-Enabled Multi-Planner Scheduling in Autonomous Vehicles
Jiawei Liu, Xun Gong, Fen Fang et al. (+6 more)
Most Human-Machine Interaction (HMI) research overlooks the maneuvering needs of passengers in autonomous driving (AD). Natural
language offers an intuitive interface, yet translating passenger open-ended instructions into control signals, without sacrificing interpretability and traceability, remains a challenge. This study proposes an instruction-realization framework that leverages a large
language model (LLM) to interpret instructions, generates executable scripts that schedule multiple model predictive control (MPC)-based motion planners based on real-time feedback, and converts planned trajectories into control signals. This scheduling-centric design decouples semantic
reasoning from vehicle control at different timescales, establishing a transparent, traceable decision-making chain from high-level instructions to low-level actions. Due to the absence of high-fidelity evaluation tools, this study introduces a
benchmark for open-ended instruction realization in a closed-loop setting. Comprehensive experiments reveal that the framework significantly improves task-completion rates over instruction-realization baselines, reduces LLM query costs, achieves safety and compliance on par with specialized AD approaches, and exhibits considerable tolerance to LLM inference latency. For more qualitative illustrations and a clearer understanding.
Read the full abstract ▾
All authors Jiawei Liu, Xun Gong, Fen Fang, Muli Yang, Bohao Qu, Yunfeng Hu, Hong Chen, Xulei Yang, Qing Guo
Robotics
AgiPIX: Bridging Simulation and Reality in Indoor Aerial Inspection
Sasanka Kuruppu Arachchige, Juan Jose Garcia, Changda Tian et al. (+4 more)
Autonomous indoor flight for critical asset inspection presents fundamental challenges in perception, planning, control, and learning. Despite rapid progress, there is still a lack of a compact, active-sensing, open-source platform that is reproducible across simulation and real-world operation. To address this gap, we present Agipix, a co-designed open hardware and software platform for indoor aerial autonomy and critical asset inspection. Agipix features a compact, hardware-synchronized active-sensing platform with onboard GPU-accelerated compute that is capable of agile flight; a containerized ROS~2-based modular autonomy stack; and a photorealistic digital twin of the hardware platform together with a reliable UI. These elements enable rapid iteration via zero-shot transfer of containerized autonomy components between simulation and real flights. We demonstrate trajectory tracking and exploration performance using onboard sensing in industrial indoor environments. All hardware designs, simulation assets, and containerized software are released openly together with documentation.
Read the full abstract ▾
All authors Sasanka Kuruppu Arachchige, Juan Jose Garcia, Changda Tian, Lauri Suomela, Panos Trahanias, Adriana Tapus, Joni-Kristian Kämäräinen
Robotics
HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation
Shuanghao Bai, Meng Li, Xinyuan Lv et al. (+15 more)
Humans achieve complex manipulation through coordinated whole-body control, whereas most
Vision-
Language-Action (VLA) models treat
robot body parts largely independently, making high-DoF
humanoid control challenging and often unstable. We present HEX, a state-centric framework for coordinated manipulation on full-sized bipedal
humanoid robots. HEX introduces a
humanoid-aligned universal state representation for scalable learning across heterogeneous embodiments, and incorporates a Mixture-of-Experts Unified Proprioceptive Predictor to model whole-body coordination and temporal motion dynamics from large-scale multi-embodiment trajectory data. To efficiently capture temporal visual context, HEX uses lightweight history
tokens to summarize past observations, avoiding repeated encoding of historical images during inference. It further employs a residual-gated fusion mechanism with a flow-matching action head to adaptively integrate visual-
language cues with proprioceptive dynamics for action generation. Experiments on real-world
humanoid manipulation tasks show that HEX achieves state-of-the-art performance in task success rate and generalization, particularly in fast-reaction and long-horizon scenarios.
Read the full abstract ▾
All authors Shuanghao Bai, Meng Li, Xinyuan Lv, Jiawei Wang, Xinhua Wang, Fei Liao, Chengkai Hou, Langzhe Gu, Bowei Zhang, Wanqi Zhou, Kun Wu, Ziluo Ding, Zhiyuan Xu, Lei Sun, Shanghang Zhang, Zhengping Che, Jian Tang, Badong Chen
Robotics · Cross-listed from eess.SY
Karma Mechanisms for Decentralised, Cooperative Multi Agent Path Finding
Kevin Riehl, Julius Schlapbach, Anastasios Kouvelas et al. (+1 more)
Multi-
Agent Path Finding (MAPF) is a fundamental coordination problem in large-scale robotic and cyber-physical systems, where multiple
agents must compute conflict-free trajectories with limited computational and communication resources. While centralised optimal solvers provide guarantees on solution optimality, their exponential computational complexity limits scalability to large-scale systems and real-time applicability. Existing decentralised heuristics are faster, but result in suboptimal outcomes and high cost disparities. This paper proposes a decentralised coordination framework for cooperative MAPF based on Karma mechanisms - artificial, non-tradeable credits that account for
agents' past cooperative behaviour and regulate future conflict resolution decisions. The approach formulates conflict resolution as a bilateral negotiation process that enables
agents to resolve conflicts through pairwise replanning while promoting long-term
fairness under limited communication and without global priority structures. The mechanism is evaluated in a lifelong robotic warehouse multi-
agent pickup-and-delivery scenario with kinematic orientation constraints. The results highlight that the Karma mechanism balances replanning effort across
agents, reducing disparity in service times without sacrificing overall efficiency. Code: https://github.com/DerKevinRiehl/karma_dmapf
Read the full abstract ▾
All authors Kevin Riehl, Julius Schlapbach, Anastasios Kouvelas, Michail A. Makridis
Robotics · Cross-listed from cs.AI
WorldMAP: Bootstrapping Vision-Language Navigation Trajectory Prediction with Generative World Models
Hongjin Chen, Shangyun Jiang, Tonghua Su et al. (+4 more)
Vision-
language models (VLMs) and generative world models are opening new opportunities for
embodied navigation. VLMs are increasingly used as direct planners or trajectory predictors, while world models support look-ahead
reasoning by imagining future views. Yet predicting a reliable trajectory from a single
egocentric observation remains challenging. Current VLMs often generate unstable trajectories, and world models, though able to synthesize plausible futures, do not directly provide the grounded signals needed for
navigation learning. This raises a central question: how can generated futures be turned into supervision for grounded trajectory prediction? We present WorldMAP, a teacher–student framework that converts world-model-generated futures into persistent semantic-spatial structure and planning-derived supervision. Its world-model-driven teacher builds semantic-spatial
memory from generated
videos, grounds task-relevant targets and obstacles, and produces trajectory pseudo-labels through explicit planning. A lightweight student with a multi-hypothesis trajectory head is then trained to predict
navigation trajectories directly from
vision-
language inputs. On Target-Bench, WorldMAP achieves the best ADE and FDE among compared methods, reducing ADE by 18.0% and FDE by 42.1% relative to the best competing baseline, while lifting a small open-source VLM to DTW performance competitive with proprietary models. More broadly, the results suggest that, in
embodied navigation, the value of world models may lie less in supplying action-ready imagined evidence than in synthesizing structured supervision for
navigation learning.
Read the full abstract ▾
All authors Hongjin Chen, Shangyun Jiang, Tonghua Su, Chen Gao, Xinlei Chen, Yong Li, Zhibo Chen
Robotics
Incremental Residual Reinforcement Learning Toward Real-World Learning for Social Navigation
Haruto Nagahisa, Kohei Matsumoto, Yuki Tomita et al. (+2 more)
As the demand for mobile
robots continues to increase, social
navigation has emerged as a critical task, driving active research into deep reinforcement learning (RL) approaches. However, because pedestrian dynamics and social conventions vary widely across different regions, simulations cannot easily encompass all possible real-world scenarios. Real-world RL, in which
agents learn while operating directly in physical environments, presents a promising solution to this issue. Nevertheless, this approach faces significant challenges, particularly regarding constrained computational resources on edge devices and learning efficiency. In this study, we propose incremental residual RL (IRRL). This method integrates incremental learning, which is a lightweight process that operates without a replay buffer or batch updates, with residual RL, which enhances learning efficiency by training only on the residuals relative to a base policy. Through the simulation experiments, we demonstrated that, despite lacking a replay buffer, IRRL achieved performance comparable to those of conventional replay buffer-based methods and outperformed existing incremental learning approaches. Furthermore, the real-world experiments confirmed that IRRL can enable
robots to effectively adapt to previously unseen environments through the real-world learning.
Read the full abstract ▾
All authors Haruto Nagahisa, Kohei Matsumoto, Yuki Tomita, Yuki Hyodo, Ryo Kurazume
Robotics
On-Policy Distillation of Language Models for Autonomous Vehicle Motion Planning
Amirhossein Afsharrad, Amirhesam Abedsoltan, Ahmadreza Moradipari et al. (+1 more)
Large
language models (LLMs) have recently demonstrated strong potential for autonomous vehicle motion planning by reformulating trajectory prediction as a
language generation problem. However, deploying capable LLMs in resource-constrained onboard systems remains a fundamental challenge. In this paper, we study how to effectively transfer motion planning knowledge from a large teacher LLM to a smaller, more deployable student model. We build on the GPT-Driver framework, which represents driving scenes as
language prompts and generates waypoint trajectories with chain-of-thought
reasoning, and investigate two student training paradigms: (i) on-policy generalized knowledge
distillation (GKD), which trains the student on its own self-generated outputs using dense
token-level feedback from the teacher, and (ii) a dense-feedback reinforcement learning (RL) baseline that uses the teacher's log-probabilities as per-
token reward signals in a policy gradient framework. Experiments on the nuScenes
benchmark show that GKD substantially outperforms the RL baseline and closely approaches teacher-level performance despite a 5$\times$ reduction in model size. These results highlight the practical value of on-policy
distillation as a principled and effective approach to deploying LLM-based planners in autonomous driving systems.
Read the full abstract ▾
All authors Amirhossein Afsharrad, Amirhesam Abedsoltan, Ahmadreza Moradipari, Sanjay Lall
Robotics
RAGE-XY: RADAR-Aided Longitudinal and Lateral Forces Estimation For Autonomous Race Cars
Davide Malvezzi, Nicola Musiu, Eugenio Mascaro et al. (+2 more)
In this work, we present RAGE-XY, an extended version of RAGE, a real-time estimation framework that simultaneously infers vehicle velocity, tire slip angles, and the forces acting on the vehicle using only standard onboard sensors such as IMUs and RADARs. Compared to the original formulation, the proposed method incorporates an online RADAR calibration module, improving the accuracy of lateral velocity estimation in the presence of sensor misalignment. Furthermore, we extend the underlying vehicle model from a single-track approximation to a tricycle model, enabling the estimation of rear longitudinal tire forces in addition to lateral dynamics. We validate the proposed approach through both high-fidelity simulations and real-world experiments conducted on the EAV-24 autonomous race car, demonstrating improved accuracy and robustness in estimating both lateral and longitudinal vehicle dynamics.
Read the full abstract ▾
All authors Davide Malvezzi, Nicola Musiu, Eugenio Mascaro, Francesco Iacovacci, Marko Bertogna
Robotics
The Sustainability Gap in Robotics: A Large-Scale Survey of Sustainability Awareness in 50,000 Research Articles
Antun Skuric, Leandro Von Werra and Thomas Wolf
We present a large-scale survey of sustainability communication and motivation in robotics research. Our analysis covers nearly 50,000 open-access papers from arXiv's cs.RO category published between 2015 and early 2026. In this study, we quantify how often papers mention social, ecological, and sustainability impacts, and we analyse their
alignment with the UN Sustainable Development Goals (SDGs). The results reveal a persistent gap between the field's potential and its stated intent. While a large fraction of robotics papers can be mapped to SDG-relevant domains, explicit sustainability motivation remains remarkably low. Specifically, mentions of sustainability-related impacts are typically below 2%, explicit SDG references stay below 0.1%, and the proportion of sustainability-motivated papers remains below 5%. These trends suggest that while the field of robotics is advancing rapidly, sustainability is not yet a standard part of research framing. We conclude by proposing concrete actions for researchers, conferences, and institutions to close these awareness and motivation gaps, supporting a shift toward more intentional and responsible innovation.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
ParkSense: Where Should a Delivery Driver Park? Leveraging Idle AV Compute and Vision-Language Models
Die Hu and Henan Li
Finding parking consumes a disproportionate share of food delivery time, yet no system addresses precise parking-spot selection relative to merchant entrances. We propose ParkSense, a framework that repurposes idle compute during low-risk AV states – queuing at red lights, traffic congestion, parking-lot crawl – to run a
Vision-
Language Model (VLM) on pre-cached satellite and street view imagery, identifying entrances and legal parking zones. We formalize the Delivery-Aware Precision Parking (DAPP) problem, show that a quantized 7B VLM completes inference in 4-8 seconds on HW4-class hardware, and estimate annual per-driver income gains of 3,000-8,000 USD in the U.S. Five open research directions are identified at this unexplored intersection of autonomous driving, computer
vision, and last-mile logistics.
Read the full abstract ▾
Robotics
Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Execution
Xue Qin, Simin Luan, John See et al. (+3 more)
Embodied Agents are evolving from passive
reasoning systems into active executors that interact with tools,
robots, and physical environments. Once an
agent gains execution authority, the central challenge shifts from how to make it act to how to keep its actions governable at runtime. Existing approaches embed safety, recovery, and decision constraints inside the
agent loop, making execution control difficult to standardize, audit, and adapt across environments. We propose a runtime governance framework for policy-constrained execution that separates
agent cognition from execution oversight. Governance is externalized into a dedicated runtime layer performing policy checking, capability admission, execution monitoring, rollback, and human override. We formalize the control boundary among a persistent
Embodied Agent, modular Capability Packages, and the governance layer, and define a policy-constrained execution pipeline evaluated under controlled simulation. Over 1000 randomized trials, the framework achieves 96.2%+/-2.7% interception of unauthorized actions, reduces unsafe continuation from 100% to 22.2%+/-3.1% under runtime drift, and attains 90.7%+/-3.0% recovery success with full policy compliance. Comparison with five baselines, including AutoRT-style constitution filtering and RoboGuard-style two-stage guardrails, shows that pre-execution filtering is equally effective across governance-aware methods, while only the proposed framework provides continuous runtime detection (RVDR = 61.3% vs. 0%) and structured recovery (all p<0.001). A sensitivity sweep across the full detection range confirms a genuine detection-continuation trade-off. This work argues future
embodied systems should be designed for governable execution.
Read the full abstract ▾
All authors Xue Qin, Simin Luan, John See, Zeyd Boukhers, Cong Yang, Zhijun Li
Robotics
Evolving Skill Modules under a Fixed Planner: Versioning, Rollback, and Runtime Governance for Long-Lived Robot Systems
Xue Qin, Simin Luan, Cong Yang et al. (+1 more)
Robots deployed for long periods keep improving their skills, and each update changes a released system. We treat this as a software-lifecycle problem: a fixed decision layer dispatches versioned skill modules and a runtime layer was built to screen each action. On six robosuite tasks we report three negative results and two measurements. First, peak task success is unstable across random seeds (within one method it spans 23.3 to 73.3%), so single-run peaks cannot rank these methods. Second, the system's four modules are whole-task policies with different labels, rotated on a clock, not the phase decomposition its documentation describes. At a matched budget one such policy holds the geometry at the final step in 0.734 of episodes reaching it, averaged over seeds, against 0.023 for the rotation, with no seed overlap at four seeds per arm (exact p=0.029). An intervention isolates why: restoring the termination condition the clock replaced raises retention on every seed. Third, our shield cut violations 98 to 100% on five single-arm tasks (34.9% on the sixth) by discarding whole actions, leaving success at zero: its acceptance criterion omitted completions, so a shield that stopped the
robot scored perfectly. What survives is release machinery: a promotion gate kept all twelve injected regressions out, a rate its calibration nearly guarantees, at a 22.5% clean-candidate rejection cost; a dip detector caught nine of twelve, missing all three on one seed.
Read the full abstract ▾
All authors Xue Qin, Simin Luan, Cong Yang, Zhijun Li
Robotics
RoboAgent: Chaining Basic Capabilities for Embodied Task Planning
Peiran Xu, Jiaqi Zheng and Yadong Mu
This paper focuses on
embodied task planning, where an
agent acquires visual observations from the environment and executes atomic actions to accomplish a given task. Although recent
Vision-
Language Models (VLMs) have achieved impressive results in
multimodal understanding and
reasoning, their performance remains limited when applied to
embodied planning that involves multi-turn interaction, long-horizon
reasoning, and extended context analysis. To bridge this gap, we propose RoboAgent, a capability-driven planning pipeline in which the model actively invokes different sub-capabilities. Each capability maintains its own context, and produces intermediate
reasoning results or interacts with the environment according to the query given by a scheduler. This framework decomposes complex planning into a sequence of basic
vision-
language problems that VLMs can better address, enabling a more transparent and controllable
reasoning process. The scheduler and all capabilities are implemented with a single VLM, without relying on external tools. To train this VLM, we adopt a multi-stage paradigm that consists of: (1) behavior cloning with expert plans, (2) DAgger training using trajectories collected by the model, and (3) reinforcement learning guided by an expert policy. Across these stages, we exploit the internal information of the environment simulator to construct high-quality supervision for each capability, and we further introduce augmented and synthetic data to enhance the model's performance in more diverse scenarios. Extensive experiments on widely used
embodied task planning
benchmarks validate the effectiveness of the proposed approach. Our codes will be available at https://github.com/woyut/RoboAgent_CVPR26.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
GEAR: GEometry-motion Alternating Refinement for Articulated Object Modeling with Gaussian Splatting
Jialin Li, Bin Fu, Ruiping Wang et al. (+1 more)
High-fidelity interactive digital assets are essential for
embodied intelligence and robotic interaction, yet articulated objects remain challenging to reconstruct due to their complex structures and coupled geometry-motion relationships. Existing methods suffer from instability in geometry-motion joint optimization, while their generalization remains limited on complex multi-joint or out-of-distribution objects. To address these challenges, we propose GEAR, an EM-style alternating optimization framework that jointly models geometry and motion as interdependent components within a Gaussian Splatting representation. GEAR treats part
segmentation as a latent variable and joint motion parameters as explicit variables, alternately refining them for improved convergence and
geometric-motion consistency. To enhance part
segmentation quality without sacrificing generalization, we leverage a vanilla 2D
segmentation model to provide multi-view part priors, and employ a weakly supervised constraint to regularize the latent variable. Experiments on multiple
benchmarks and our newly constructed
dataset GEAR-Multi demonstrate that GEAR achieves state-of-the-art results in
geometric reconstruction and motion parameters estimation, particularly on complex articulated objects with multiple movable parts.
Read the full abstract ▾
All authors Jialin Li, Bin Fu, Ruiping Wang, Xilin Chen
Robotics
Xingyu Xia, Lekai Zhou, Yujie Tang et al. (+3 more)
Aerial
vision-and-
language navigation (Aerial VLN) aims to enable unmanned aerial vehicles (
UAVs) to interpret natural
language instructions and autonomously navigate complex three-dimensional environments by grounding
language in visual perception. This survey provides a critical and analytical review of the Aerial VLN field, with particular
attention to the recent integration of large
language models (LLMs) and
vision-
language models (VLMs). We first formally introduce the Aerial VLN problem and define two interaction paradigms: single-instruction and dialog-based, as foundational axes. We then organize the body of Aerial VLN methods into a taxonomy of five architectural categories: sequence-to-sequence and
attention-based methods, end-to-end LLM/VLM methods, hierarchical methods, multi-
agent methods, and dialog-based
navigation methods. For each category, we systematically analyze design rationales, technical trade-offs, and reported performance. We critically assess the evaluation infrastructure for Aerial VLN, including
datasets, simulation platforms, and metrics, and identify their gaps in scale, environmental diversity, real-world grounding, and metric coverage. We consolidate cross-method comparisons on shared
benchmarks and analyze key architectural trade-offs, including discrete versus continuous actions, end-to-end versus hierarchical designs, and the simulation-to-reality gap. Finally, we synthesize seven concrete open problems: long-horizon instruction grounding, viewpoint robustness, scalable spatial representation, continuous 6-DoF action execution, onboard deployment,
benchmark standardization, and multi-
UAV swarm
navigation, with specific research directions grounded in the evidence presented throughout the survey.
Read the full abstract ▾
All authors Xingyu Xia, Lekai Zhou, Yujie Tang, Xiaozhou Zhu, Hai Zhu, Wen Yao
Robotics
Bird-Inspired Spatial Flapping Wing Mechanism via Coupled Linkages with Single Actuator
Daniel Huczala, Sun-Pill Jung and Frank C. Park
Spatial single-loop mechanisms such as Bennett linkages offer a unique combination of one-degree-of-freedom actuation and nontrivial spatial trajectories, making them attractive for lightweight bio-inspired robotic design. However, although they appear simple and elegant, the
geometric task-based synthesis is rather complicated and often avoided in engineering tasks due to the mathematical complexity involved. This paper presents a bird-inspired flapping-wing mechanism built from two coupled spatial four-bars, driven by a single motor. One linkage is actuated to generate the desired spatial sweeping stroke, while the serially coupled linkage remains unactuated and passively switches between extended and folded wing configurations over the stroke cycle. We introduce a simplified kinematic methodology for constructing Bennett linkages from quadrilaterals that contain a desired surface area and further leverage mechanically induced passive state switching. This architecture realizes a coordinated sweep-and-fold wing motion with a single actuation input, reducing weight and control complexity. A 3D-printed prototype is assembled and tested, demonstrating the intended spatial stroke and passive folding behavior.
Read the full abstract ▾
Robotics
ReBound: Reset-Free Reinforcement Learning for Agile Driving via Reset-Aware Semi-Markov Bootstrapping
Kohei Honda, Aiki Kada and Hirotaka Hosogaya
We present ReBound (reset-aware semi-Markov bootstrapping), a value-learning method for reset-free reinforcement learning (RL) that learns agile driving through real-world RL without human intervention after training starts. Reset-free RL explores by alternating between a forward policy that performs the task and a reset policy that restores a drivable state. Standard reset-free RL treats collisions as episode terminations in value-function bootstrapping, truncating the return at each collision. This conflicts with the reset-free objective, under which driving resumes after recovery, and acts as a hidden collision penalty absent from the reward function. ReBound instead applies semi-Markov bootstrapping: it treats the collision-causing action and the subsequent recovery as a single semi-Markov transition and bootstraps from the post-recovery value, discounted by the actual recovery time. We further apply reward centering to stabilize value learning in the resulting continuing task. In simulation and on a 1/10-scale vehicle, ReBound substantially outperforms conventional value-learning methods and model-based control.
Read the full abstract ▾