Robotics
ReGen: Generative Robot Simulation via Inverse Design
Phat Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong et al. (+5 more)
Simulation plays a key role in scaling
robot learning and validating policies, but constructing simulations remains a labor-intensive process. This paper introduces ReGen, a generative simulation framework that automates simulation design via inverse design. Given a
robot's behavior – such as a motion trajectory or an objective function – and its textual description, ReGen infers plausible scenarios and environments that could have caused the behavior. ReGen leverages large
language models to synthesize scenarios by expanding a directed graph that encodes cause-and-effect relationships, relevant entities, and their properties. This structured graph is then translated into a symbolic program, which configures and executes a
robot simulation environment. Our framework supports (i) augmenting simulations based on ego-
agent behaviors, (ii) controllable, counterfactual scenario generation, (iii)
reasoning about
agent cognition and mental states, and (iv)
reasoning with distinct sensing modalities, such as braking due to faulty GPS signals. We demonstrate ReGen in autonomous driving and
robot manipulation tasks, generating more diverse, complex simulated environments compared to existing simulations with high success rates, and enabling controllable generation for corner cases. This approach enhances the validation of
robot policies and supports data or simulation augmentation, advancing scalable
robot learning for improved generalization and robustness. We provide code and example
videos at: https://regen-sim.github.io/
Read the full abstract ▾
All authors Phat Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong, Erfan Aasi, Andrew Silva, Guy Rosman, Sertac Karaman, Daniela Rus
Robotics
ScheduleStream: Temporal Planning with Samplers for GPU-Accelerated Multi-Arm Task and Motion Planning & Scheduling
Caelan Garrett and Fabio Ramos
Bimanual and
humanoid robots are appealing because of their human-like ability to leverage multiple arms to efficiently complete tasks. However, controlling multiple arms at once is computationally challenging due to the growth in the hybrid discrete-continuous action space. Task and Motion Planning (TAMP)
algorithms can efficiently plan in hybrid spaces but generally produce plans, where only one arm is moving at a time, rather than schedules that allow for parallel arm motion. In order to extend TAMP to produce schedules, we present ScheduleStream, the first general-purpose framework for planning & scheduling with sampling operations. ScheduleStream models temporal dynamics using hybrid durative actions, which can be started asynchronously and persist for a duration that's a function of their parameters. We propose domain-independent
algorithms that solve ScheduleStream problems without any application-specific mechanisms. We apply ScheduleStream to Task and Motion Planning & Scheduling (TAMPAS), where we use GPU acceleration within samplers to expedite planning. We compare ScheduleStream
algorithms to several ablations in simulation and find that they produce more efficient solutions. We demonstrate ScheduleStream on several real-world bimanual
robot tasks at https://schedulestream.github.io.
Read the full abstract ▾
Robotics
GentleHumanoid: Learning Upper-body Compliance for Contact-rich Human and Object Interaction
Qingzhou Lu, Yao Feng, Baiyu Shi et al. (+3 more)
Humanoid robots are expected to operate in human-centered environments where safe and natural physical interaction is essential. However, most recent reinforcement learning (RL) policies emphasize rigid tracking and suppress external forces. Existing impedance-augmented approaches are typically restricted to base or end-effector control and focus on resisting extreme forces rather than enabling compliance. We introduce GentleHumanoid, a framework that integrates impedance control into a whole-body motion tracking policy to achieve upper-body compliance. At its core is a unified spring-based formulation that models both resistive contacts (restoring forces when pressing against surfaces) and guiding contacts (pushes or pulls sampled from human motion data). This formulation ensures kinematically consistent forces across the shoulder, elbow, and wrist, while exposing the policy to diverse interaction scenarios. Safety is further supported through task-adjustable force thresholds. We evaluate our approach in both simulation and on the Unitree G1
humanoid across tasks requiring different levels of compliance, including gentle hugging, sit-to-stand assistance, and safe object manipulation. Compared to baselines, our policy consistently reduces peak contact forces while maintaining task success, resulting in smoother and more natural interactions. These results highlight a step toward
humanoid robots that can safely and effectively collaborate with humans and handle objects in real-world environments.
Read the full abstract ▾
All authors Qingzhou Lu, Yao Feng, Baiyu Shi, Michael Piseno, Zhenan Bao, C. Karen Liu
Robotics
X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations
Maximus A. Pace, Prithwish Dan, Chuanruo Ning et al. (+5 more)
Human
videos are a scalable source of training data for
robot learning. However, humans and
robots significantly differ in embodiment, making many human actions infeasible for direct execution on a
robot. Still, these demonstrations convey rich object-interaction cues and task intent. Our goal is to learn from this coarse guidance without transferring embodiment-specific, infeasible execution strategies. Recent advances in generative modeling tackle a related problem of learning from low-quality data. In particular, Ambient
Diffusion is a recent method for
diffusion modeling that incorporates low-quality data only at high-noise timesteps of the forward
diffusion process. Our key insight is to view human actions as noisy counterparts of
robot actions. As noise increases along the forward
diffusion process, embodiment-specific differences fade away while task-relevant guidance is preserved. Based on these observations, we present X-
Diffusion, a cross-embodiment learning framework based on Ambient
Diffusion that selectively trains
diffusion policies on noised human actions. This enables effective use of easy-to-collect human
videos without sacrificing
robot feasibility. Across five real-world manipulation tasks, we show that X-
Diffusion improves average success rates by 16% over naive co-training and manual data filtering. The project website is available at https://portal-cornell.github.io/X-
Diffusion/.
Read the full abstract ▾
All authors Maximus A. Pace, Prithwish Dan, Chuanruo Ning, Atiksh Bhardwaj, Audrey Du, Edward W. Duan, Wei-Chiu Ma, Kushal Kedia
Robotics
Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions
Kaifeng Zhang, Shuo Sha, Hanxiao Jiang et al. (+7 more)
Robotic manipulation policies are advancing rapidly, but their direct evaluation in the real world remains costly, time-consuming, and difficult to reproduce, particularly for tasks involving deformable objects. Simulation provides a scalable and systematic alternative, yet existing simulators often fail to capture the coupled visual and physical complexity of soft-body interactions. We present a real-to-sim policy evaluation framework that constructs soft-body digital twins from real-world
videos and renders
robots, objects, and environments with photorealistic fidelity using 3D Gaussian Splatting. We validate our approach on representative deformable manipulation tasks, including plush toy packing, rope routing, and T-block pushing, demonstrating that simulated rollouts correlate strongly with real-world execution performance and reveal key behavioral patterns of learned policies. Our results suggest that combining physics-informed reconstruction with high-quality rendering enables reproducible, scalable, and accurate evaluation of robotic manipulation policies. Website: https://real2sim-eval.github.io/
Read the full abstract ▾
All authors Kaifeng Zhang, Shuo Sha, Hanxiao Jiang, Matthew Loper, Hyunjong Song, Guangyan Cai, Zhuo Xu, Xiaochen Hu, Changxi Zheng, Yunzhu Li
Robotics
SAFe-Copilot: Unified Shared Autonomy Framework
Phat Nguyen, Erfan Aasi, Shiva Sreeram et al. (+4 more)
Autonomous driving systems remain brittle in rare, ambiguous, and out-of-distribution scenarios, where human driver succeed through contextual
reasoning. Shared autonomy has emerged as a promising approach to mitigate such failures by incorporating human input when autonomy is uncertain. However, most existing methods restrict arbitration to low-level trajectories, which represent only
geometric paths and therefore fail to preserve the underlying driving intent. We propose a unified shared autonomy framework that integrates human input and autonomous planners at a higher level of abstraction. Our method leverages
Vision Language Models (VLMs) to infer driver intent from multi-modal cues – such as driver actions and environmental context – and to synthesize coherent strategies that mediate between human and autonomous control. We first study the framework in a mock-human setting, where it achieves perfect recall alongside high accuracy and precision. A human-subject survey further shows strong
alignment, with participants agreeing with arbitration outcomes in 92% of cases. Finally, evaluation on the Bench2Drive
benchmark demonstrates a substantial reduction in collision rate and improvement in overall performance compared to pure autonomy. Arbitration at the level of semantic,
language-based representations emerges as a design principle for shared autonomy, enabling systems to exercise common-sense
reasoning and maintain continuity with human intent.
Read the full abstract ▾
All authors Phat Nguyen, Erfan Aasi, Shiva Sreeram, Guy Rosman, Andrew Silva, Sertac Karaman, Daniela Rus
Robotics
Evo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignment
Tao Lin, Yilei Zhong, Yuxin Du et al. (+11 more)
Vision-
Language-Action (VLA) models have emerged as a powerful framework that unifies perception,
language, and control, enabling
robots to perform diverse tasks through
multimodal understanding. However, current VLA models typically contain massive parameters and rely heavily on large-scale
robot data pretraining, leading to high computational costs during training, as well as limited deployability for real-time inference. Moreover, most training paradigms often degrade the perceptual representations of the
vision-
language backbone, resulting in overfitting and poor generalization to downstream tasks. In this work, we present Evo-1, a lightweight VLA model that reduces computation and improves deployment efficiency, while maintaining strong performance without pretraining on
robot data. Evo-1 builds on a native
multimodal Vision-
Language model (VLM), incorporating a novel cross-modulated
diffusion transformer along with an optimized integration module, together forming an effective architecture. We further introduce a two-stage training paradigm that progressively aligns action with perception, preserving the representations of the VLM. Notably, with only 0.77 billion parameters, Evo-1 achieves state-of-the-art results on the Meta-World and RoboTwin suite, surpassing the previous best models by 12.4% and 6.9%, respectively, and also attains a competitive result of 94.8% on LIBERO. In real-world evaluations, Evo-1 attains a 78% success rate with high inference frequency and low
memory overhead, outperforming all baseline methods. We release code, data, and model weights to facilitate future research on lightweight and efficient VLA models.
Read the full abstract ▾
All authors Tao Lin, Yilei Zhong, Yuxin Du, Jingjing Zhang, Jiting Liu, Yinxinyu Chen, Encheng Gu, Ziyan Liu, Hongyi Cai, Yanwen Zou, Lixing Zou, Zhaoye Zhou, Gen Li, Bo Zhao
Robotics
Temporal Action Selection for Action Chunking
Yueyang Weng, Xiaopeng Zhang, Yongjin Mu et al. (+2 more)
Action chunking is a widely adopted approach in Learning from Demonstration (LfD). By modeling multi-step action chunks rather than single-step actions, action chunking significantly enhances modeling capabilities for human expert policies. However, because action chunking makes a single decision only after a complete action block has been executed, the resulting reduction in decision frequency restricts the utilization of real-time observations, impairing reactivity in dynamic or noisy environments. Existing efforts to address this issue have primarily resorted to trading off reactivity against decision consistency, without achieving both. To address this limitation, we propose a novel
algorithm, Temporal Action Selection (TAS), which caches predicted action chunks from multiple timesteps and dynamically selects the optimal action through a lightweight selector network. TAS achieves balanced optimization across both reactivity and decision consistency. Experiments across multiple tasks with diverse base policy architectures show that TAS significantly improves success rates. Furthermore, integrating TAS as a base policy with residual reinforcement learning (RL) improves both training efficiency and the performance ceiling. Experiments in both simulation and physical
robots confirm the method's efficacy.
Read the full abstract ▾
All authors Yueyang Weng, Xiaopeng Zhang, Yongjin Mu, Yingcong Zhu, Yanjie Li
Robotics · Cross-listed from cs.CV
BoRe-Depth: Self-supervised Monocular Depth Estimation with Boundary Refinement for Embedded Systems
Chang Liu, Juan Li, Sheng Zhang et al. (+3 more)
Depth estimation is one of the key technologies for realizing 3D perception in unmanned systems.
Monocular depth estimation has been widely researched because of its low-cost advantage, but the existing methods face the challenges of poor depth estimation performance and blurred object boundaries on embedded systems. In this paper, we propose a novel
monocular depth estimation model, BoRe-Depth, which contains only 8.7M parameters. It can accurately estimate depth maps on embedded systems and significantly improves boundary quality. Firstly, we design an Enhanced Feature Adaptive Fusion Module (EFAF) which adaptively fuses depth features to enhance boundary detail representation. Secondly, we integrate semantic knowledge into the encoder to improve the object recognition and boundary perception capabilities. Finally, BoRe-Depth is deployed on NVIDIA Jetson Orin, and runs efficiently at 50.7 FPS. We demonstrate that the proposed model significantly outperforms previous lightweight models on multiple challenging
datasets, and we provide detailed ablation studies for the proposed methods. The code is available at https://github.com/liangxiansheng093/BoRe-Depth.
Read the full abstract ▾
All authors Chang Liu, Juan Li, Sheng Zhang, Chang Liu, Jie Li, Xu Zhang
Robotics
ForeRobo: Unlocking Infinite Simulation Data for 3D Goal-driven Robotic Manipulation
Dexin wang, Faliang Chang and Chunsheng Liu
Efficiently leveraging simulation to acquire advanced manipulation skills is both challenging and highly significant. We introduce ForeRobo, a generative robotic
agent that utilizes generative simulations to autonomously acquire manipulation skills driven by envisioned goal states. Instead of directly learning low-level policies, we advocate integrating generative paradigms with classical control. Our approach equips a robotic
agent with a self-guided propose-generate-learn-actuate cycle. The
agent first proposes the skills to be acquired and constructs the corresponding simulation environments; it then configures objects into appropriate arrangements to generate skill-consistent goal states (ForeGen). Subsequently, the virtually infinite data produced by ForeGen are used to train the proposed state generation model (ForeFormer), which establishes point-wise correspondences by predicting the 3D goal position of every point in the current state, based on the scene state and task instructions. Finally, classical control
algorithms are employed to drive the
robot in real-world environments to execute actions based on the envisioned goal states. Compared with end-to-end policy learning methods, ForeFormer offers superior interpretability and execution efficiency. We train and
benchmark ForeFormer across a variety of rigid-body and articulated-object manipulation tasks, and observe an average improvement of 56.32% over the state-of-the-art state generation models, demonstrating strong generality across different manipulation patterns. Moreover, in real-world evaluations involving more than 20 robotic tasks, ForeRobo achieves zero-shot sim-to-real transfer and exhibits remarkable generalization capabilities, attaining an average success rate of 79.28%.
Read the full abstract ▾
Robotics
Studying the Effect of Explicit Interaction Representations on Learning Scene-level Distributions of Human Trajectories
Anna Mészáros, Javier Alonso-Mora and Jens Kober
Effectively capturing the joint distribution of all
agents in a scene is relevant for predicting the true
evolution of the scene and in turn providing more accurate information to the decision processes of autonomous vehicles. While new models have been developed for this purpose in recent years, it remains unclear how to best represent the joint distributions particularly from the perspective of the interactions between
agents. Thus far there is no clear consensus on how best to represent interactions between
agents; whether they should be learned implicitly from data by neural networks, or explicitly modeled using the spatial and temporal relations that are more grounded in human decision-making. This paper aims to study various means of describing interactions within the same network structure and their effect on the final learned joint distributions. Our findings show that more often than not, simply allowing a network to establish interactive connections between
agents based on data has a detrimental effect on performance. Instead, having well defined interactions (such as which
agent of an
agent pair passes first at an intersection) can often bring about a clear boost in performance.
Read the full abstract ▾
Robotics
GraSP-VLA: Graph-based Symbolic Action Representation for Long-Horizon Planning with VLA Policies
Maëlic Neau, Zoe Falomir, Paulo E. Santos et al. (+2 more)
Deploying autonomous
robots that can learn new skills from demonstrations is an important challenge of modern robotics. Existing solutions often apply end-to-end imitation learning with
Vision-
Language Action (VLA) models or symbolic approaches with Action Model Learning (AML). On the one hand, current VLA models are limited by the lack of high-level symbolic planning, which hinders their abilities in long-horizon tasks. On the other hand, symbolic approaches in AML lack generalization and scalability perspectives. In this paper we present a new neuro-symbolic approach, GraSP-VLA, a framework that uses a Continuous Scene Graph representation to generate a symbolic representation of human demonstrations. This representation is used to generate new planning domains during inference and serves as an orchestrator for low-level VLA policies, scaling up the number of actions that can be reproduced in a row. Our results show that GraSP-VLA is effective for modeling symbolic representations on the task of automatic planning domain generation from observations. In addition, results on real-world experiments show the potential of our Continuous Scene Graph representation to orchestrate low-level VLA policies in long-horizon tasks.
Read the full abstract ▾
All authors Maëlic Neau, Zoe Falomir, Paulo E. Santos, Anne-Gwenn Bosser, Cédric Buche
Robotics
MacroNav: Multi-Task Context Representation Learning Enables Efficient Navigation in Unknown Environments
Kuankuan Sima, Longbin Tang, Zhenyu Yang et al. (+2 more)
Autonomous
navigation in unknown environments requires multi-scale spatial understanding that captures
geometric details, topological connectivity, and global structure to support high-level decision making under partial observability. Existing approaches struggle to efficiently capture such multi-scale spatial understanding while maintaining low computational cost for real-time
navigation. We present MacroNav, a learning-based
navigation framework featuring two key components: (1) a lightweight context encoder trained via multi-task self-supervised learning to capture multi-scale,
navigation-centric spatial representations; and (2) a reinforcement learning policy that seamlessly integrates these representations with graph-based
reasoning for efficient action selection. Extensive experiments demonstrate the context encoder's effective and robust environmental understanding. Real-world deployments further validate MacroNav's effectiveness, yielding significant gains over state-of-the-art
navigation methods in both Success Rate (SR) and Success weighted by Path Length (SPL), with superior computational efficiency.
Read the full abstract ▾
All authors Kuankuan Sima, Longbin Tang, Zhenyu Yang, Haozhe Ma, Lin Zhao
Robotics
Design and Control of a Coaxial Dual-rotor Reconfigurable Tailsitter UAV Based on Swashplateless Mechanism
Jinfeng Liang, Haocheng Guo and Ximin Lyu
The tailsitter vertical takeoff and landing (VTOL)
UAV is widely used due to its lower dead weight, which eliminates the actuators and mechanisms for tilting. However, the tailsitter
UAV is susceptible to wind disturbances in multi-rotor mode, as it exposes a large frontal fuselage area. To address this issue, our tailsitter
UAV features a reconfigurable wing design, allowing wings to retract in multi-rotor mode and extend in fixed- wing mode. Considering power efficiency, we design a coaxial heterogeneous dual-rotor configuration, which significantly re- duces the total power consumption. To reduce structural weight and simplify structural complexity, we employ a swashplateless mechanism with an improved design to control pitch and roll in multi-rotor mode. We optimize the structure of the swashplateless mechanism by adding flapping hinges, which reduces vibration during cyclic acceleration and deceleration. Finally, we perform comprehensive transition flight tests to validate stable flight performance across the entire flight envelope of the tailsitter
UAV.
Read the full abstract ▾
Robotics
Can Context Bridge the Reality Gap? Sim-to-Real Transfer of Context-Aware Policies
Marco Iannotta, Yuxuan Yang, Johannes A. Stork et al. (+2 more)
Sim-to-real transfer remains a major challenge in reinforcement learning (RL) for robotics, as policies trained in simulation often fail to generalize to the real world due to discrepancies in environment dynamics. Domain Randomization (DR) mitigates this issue by exposing the policy to a wide range of randomized dynamics during training, yet leading to a reduction in performance. While standard approaches typically train policies agnostic to these variations, we investigate whether sim-to-real transfer can be improved by conditioning the policy on an estimate of the dynamics parameters – referred to as context. To this end, we integrate a context estimation module into a DR-based RL framework and systematically compare SOTA supervision strategies. We evaluate the resulting context-aware policies in both a canonical control
benchmark and a real-world pushing task using a Franka Emika Panda
robot. Results show that context-aware policies outperform the context-agnostic baseline across all settings, although the best supervision strategy depends on the task.
Read the full abstract ▾
All authors Marco Iannotta, Yuxuan Yang, Johannes A. Stork, Erik Schaffernicht, Todor Stoyanov
Robotics
GraspView: Active Perception Scoring and Best-View Optimization for Robotic Grasping in Cluttered Environments
Shenglin Wang, Mingtong Dai, Jingxuan Su et al. (+4 more)
Robotic grasping is a fundamental capability for autonomous manipulation, yet remains highly challenging in cluttered environments where occlusion, poor perception quality, and inconsistent 3D reconstructions often lead to unstable or failed grasps. Conventional pipelines have widely relied on RGB-D cameras to provide
geometric information, which fail on transparent or glossy objects and degrade at close range. We present GraspView, an RGB-only robotic grasping pipeline that achieves accurate manipulation in cluttered environments without depth sensors. Our framework integrates three key components: (i) global perception scene reconstruction, which provides locally consistent, up-to-scale geometry from a single RGB view and fuses multi-view projections into a coherent global 3D scene; (ii) a render-and-score active perception strategy, which dynamically selects next-best-views to reveal occluded regions; and (iii) an online metric
alignment module that calibrates VGGT predictions against
robot kinematics to ensure physical scale consistency. Building on these tailor-designed modules, GraspView performs best-view global grasping, fusing multi-view reconstructions and leveraging GraspNet for robust execution. Experiments on diverse tabletop objects demonstrate that GraspView significantly outperforms both RGB-D and single-view RGB baselines, especially under heavy occlusion, near-field sensing, and with transparent objects. These results highlight GraspView as a practical and versatile alternative to RGB-D pipelines, enabling reliable grasping in unstructured real-world environments.
Read the full abstract ▾
All authors Shenglin Wang, Mingtong Dai, Jingxuan Su, Lingbo Liu, Chunjie Chen, Xinyu Wu, Liang Lin
Robotics
PUL-SLAM: Path-Uncertainty Co-Optimization with Lightweight Stagnation Detection for Efficient Robotic Exploration
Yizhen Yin, Dapeng Feng, Hongbo Chen et al. (+1 more)
Existing Active SLAM methodologies face issues such as slow exploration speed and suboptimal paths. To address these limitations, we propose a hybrid framework combining a Path-Uncertainty Co-Optimization Deep Reinforcement Learning framework and a Lightweight Stagnation Detection mechanism. The Path-Uncertainty Co-Optimization framework jointly optimizes travel distance and map uncertainty through a dual-objective reward function, balancing exploration and exploitation. The Lightweight Stagnation Detection reduces redundant exploration through
Lidar Static Anomaly Detection and Map Update Stagnation Detection, terminating episodes on low expansion rates. Experimental results show that compared with the frontier-based method and RRT method, our approach shortens exploration time by up to 65% and reduces path distance by up to 42%, significantly improving exploration efficiency in complex environments while maintaining reliable map completeness. Ablation studies confirm that the collaborative mechanism accelerates training convergence. Empirical validation on a physical robotic platform demonstrates the
algorithm's practical applicability and its successful transferability from simulation to real-world environments.
Read the full abstract ▾
All authors Yizhen Yin, Dapeng Feng, Hongbo Chen, Yuhua Qi
Robotics
BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning
Yitang Li, Zhengyi Luo, Tonghe Zhang et al. (+10 more)
Building Behavioral Foundation Models (BFMs) for
humanoid robots has the potential to unify diverse control tasks under a single, promptable generalist policy. However, existing approaches are either exclusively deployed on simulated
humanoid characters, or specialized to specific tasks such as tracking. We propose BFM-Zero, a framework that learns an effective shared latent representation that embeds motions, goals, and rewards into a common space, enabling a single policy to be prompted for multiple downstream tasks without retraining. This well-structured latent space in BFM-Zero enables versatile and robust whole-body skills on a Unitree G1
humanoid in the real world, via diverse inference methods, including zero-shot motion tracking, goal reaching, and reward optimization, and few-shot optimization-based adaptation. Unlike prior on-policy reinforcement learning (RL) frameworks, BFM-Zero builds upon recent advancements in unsupervised RL and Forward-Backward (FB) models, which offer an objective-centric, explainable, and smooth latent representation of whole-body motions. We further extend BFM-Zero with critical reward shaping, domain randomization, and history-dependent asymmetric learning to bridge the sim-to-real gap. Those key design choices are quantitatively ablated in simulation. A first-of-its-kind model, BFM-Zero establishes a step toward scalable, promptable behavioral foundation models for whole-body
humanoid control.
Read the full abstract ▾
All authors Yitang Li, Zhengyi Luo, Tonghe Zhang, Cunxi Dai, Anssi Kanervisto, Andrea Tirinzoni, Haoyang Weng, Kris Kitani, Mateusz Guzek, Ahmed Touati, Alessandro Lazaric, Matteo Pirotta, Guanya Shi
Robotics
CBMC-V3: A CNS-inspired Control Framework Towards Agile Manipulation with SNN
Yanbo Pang, Qingkai Li and Mingguo Zhao
As robotic arm applications expand beyond traditional industrial settings into service-oriented domains such as catering, household and retail, existing control
algorithms struggle to achieve the level of agile manipulation required in unstructured environments characterized by dynamic trajectories, unpredictable interactions, and diverse objects. This paper presents a biomimetic control framework based on Spiking Neural Network (SNN), inspired by the human Central Nervous System (CNS), to address these challenges. The proposed framework comprises five control modules-cerebral cortex, cerebellum, thalamus, brainstem, and spinal cord-organized into three hierarchical control levels (first-order, second-order, and third-order) and two information pathways (ascending and descending). All modules are fully implemented using SNN. The framework is validated through both simulation and experiments on a commercial robotic arm platform across a range of control tasks. The results demonstrate that the proposed method outperforms the baseline in terms of agile motion control capability, offering a practical and effective solution for achieving agile manipulation.
Read the full abstract ▾
Robotics
Enhancing Fault-Tolerant Space Computing: Guidance Navigation and Control (GNC) and Landing Vision System (LVS) Implementations on Next-Gen Multi-Core Processors
Kyongsik Yun, David Bayard, Gerik Kubiak et al. (+5 more)
Future planetary exploration missions demand high-performance, fault-tolerant computing to enable autonomous Guidance,
Navigation, and Control (GNC) and Lander
Vision System (LVS) operations during Entry, Descent, and Landing (EDL). This paper evaluates the deployment of GNC and LVS
algorithms on next-generation multi-core processors–HPSC, Snapdragon VOXL2, and AMD Xilinx Versal–demonstrating up to 15x speedup for LVS image processing and over 250x speedup for Guidance for Fuel-Optimal Large Divert (GFOLD) trajectory optimization compared to legacy spaceflight hardware. To ensure computational reliability, we present ARBITER (Asynchronous Redundant Behavior Inspection for Trusted Execution and Recovery), a Multi-Core Voting (MV) mechanism that performs real-time fault detection and correction across redundant cores. ARBITER is validated in both static optimization tasks (GFOLD) and dynamic closed-loop control (Attitude Control System). A fault injection study further identifies the gradient computation stage in GFOLD as the most sensitive to bit-level errors, motivating selective protection strategies and vector-based output arbitration. This work establishes a scalable and energy-efficient architecture for future missions, including Mars Sample Return, Enceladus Orbilander, and Ceres Sample Return, where onboard autonomy, low latency, and fault resilience are critical.
Read the full abstract ▾
All authors Kyongsik Yun, David Bayard, Gerik Kubiak, Austin Owens, Andrew Johnson, Ryan Johnson, Dan Scharf, Thomas Lu
Robotics
An LLM-based Framework for Human-Swarm Teaming Cognition in Disaster Search and Rescue
Kailun Ji, Xiaoyu Hu, Xinyu Zhang et al. (+1 more)
Large-scale disaster Search And Rescue (SAR) operations are persistently challenged by complex terrain and disrupted communications. While Unmanned Aerial Vehicle (
UAV) swarms offer a promising solution for tasks like wide-area search and supply delivery, yet their effective coordination places a significant cognitive burden on human operators. The core human-machine collaboration bottleneck lies in the “intention-to-action gap”, which is an error-prone process of translating a high-level rescue objective into a low-level swarm command under high intensity and pressure. To bridge this gap, this study proposes a novel LLM-CRF system that leverages Large
Language Models (LLMs) to model and augment human-swarm teaming cognition. The proposed framework initially captures the operator's intention through natural and multi-modal interactions with the device via voice or graphical annotations. It then employs the LLM as a cognitive engine to perform intention comprehension, hierarchical task decomposition, and mission planning for the
UAV swarm. This closed-loop framework enables the swarm to act as a proactive partner, providing active feedback in real-time while reducing the need for manual monitoring and control, which considerably advances the efficacy of the SAR task. We evaluate the proposed framework in a simulated SAR scenario. Experimental results demonstrate that, compared to traditional order and command-based interfaces, the proposed LLM-driven approach reduced task completion time by approximately $64.2\%$ and improved task success rate by $7\%$. It also leads to a considerable reduction in subjective cognitive workload, with NASA-TLX scores dropping by $42.9\%$. This work establishes the potential of LLMs to create more intuitive and effective human-swarm collaborations in high-stakes scenarios.
Read the full abstract ▾
All authors Kailun Ji, Xiaoyu Hu, Xinyu Zhang, Jun Chen
Robotics
Integrating Ergonomics and Manipulability for Upper Limb Postural Optimization in Bimanual Human-Robot Collaboration
Chenzui Li, Yiming Chen, Xi Wu et al. (+2 more)
This paper introduces an upper limb postural optimization method for enhancing physical ergonomics and force manipulability during bimanual human-
robot co-carrying tasks. Existing research typically emphasizes human safety or manipulative efficiency, whereas our proposed method uniquely integrates both aspects to strengthen collaboration across diverse conditions (e.g., different grasping postures of humans, and different shapes of objects). Specifically, the joint angles of a simplified human skeleton model are optimized by minimizing the cost function to prioritize safety and manipulative capability. To guide humans towards the optimized posture, the reference end-effector poses of the
robot are generated through a transformation module. A bimanual model predictive impedance controller (MPIC) is proposed for our human-like
robot, CURI, to recalibrate the end effector poses through planned trajectories. The proposed method has been validated through various subjects and objects during human-human collaboration (HHC) and human-
robot collaboration (HRC). The experimental results demonstrate significant improvement in muscle conditions by comparing the activation of target muscles before and after optimization.
Read the full abstract ▾
All authors Chenzui Li, Yiming Chen, Xi Wu, Giacinto Barresi, Fei Chen
Robotics
Learning Vision-Driven Reactive Soccer Skills for Humanoid Robots
Yushi Wang, Changsheng Luo, Penghui Chen et al. (+7 more)
Humanoid soccer poses a representative challenge for
embodied intelligence, requiring
robots to coordinate agile locomotion with unreliable visual perception in dynamic environments. However, existing systems typically rely on modular pipelines that separate perception from control or assume ideal sensing, making it difficult to achieve coherent and reactive behavior under real-world perceptual limitations. In this work, we present a unified reinforcement learning-based controller that enables
humanoid robots to learn
vision-driven reactive soccer skills by directly coupling visual perception with locomotion control. The
robot is trained in simulation to acquire soccer behaviors, and
adversarial motion priors guide policy learning toward natural motion patterns. To support robust performance under imperfect sensing, we introduce an encoder-decoder architecture together with a virtual perception system that models key characteristics of onboard
vision, exposing the policy to perceptual noise and detection failures during training. This design encourages the policy to internalize perceptual uncertainty and continuously adapt its motion in a closed loop. The resulting controller produces coordinated soccer behaviors using only onboard
vision, including ball searching, chasing, and multidirectional kicking. It reduces ball position estimation error by 46% and shortens time-to-kick by up to 64% compared with a rule-based baseline, achieving around 90% kicking success in frontfield positions. Experiments across diverse environments and dynamic scenarios, including real RoboCup competitions, further demonstrate the robust performance of the controller. These results highlight the practical effectiveness of integrating perceptual uncertainty directly into policy learning for achieving reliable
vision-driven behaviors in
humanoid robots operating under real-world conditions.
Read the full abstract ▾
All authors Yushi Wang, Changsheng Luo, Penghui Chen, Jianran Liu, Weijian Sun, Tong Guo, Kechang Yang, Biao Hu, Yangang Zhang, Mingguo Zhao
Robotics
Dynamic Shape Control of Soft Robots Enabled by Data-Driven Model Reduction
Iman Adibnazari, Harsh Sharma, Myungsun Park et al. (+3 more)
Soft
robots have shown immense promise in settings where they can leverage dynamic control of their entire bodies. However, effective dynamic shape control requires a controller that accounts for the
robot's high-dimensional dynamics–a challenge exacerbated by a lack of general-purpose tools for modeling soft
robots amenably for control. In this work, we conduct a comparative study of data-driven model reduction techniques for generating linear models amendable to dynamic shape control. We focus on three methods–the eigensystem realization
algorithm, dynamic mode decomposition with control, and the Lagrangian operator inference (LOpInf) method. Using each class of model, we explored their efficacy in model predictive control policies for the dynamic shape control of a simulated eel-inspired soft
robot in three experiments: 1) tracking simulated reference trajectories guaranteed to be feasible, 2) tracking reference trajectories generated from a biological model of eel kinematics, and 3) tracking reference trajectories generated by a reduced-scale physical analog. In all experiments, the LOpInf-based policies generated lower tracking errors than policies based on other models.
Read the full abstract ▾
All authors Iman Adibnazari, Harsh Sharma, Myungsun Park, Jacobo Cervera-Torralba, Boris Kramer, Michael T. Tolley