Robotics
A Hazard-Informed Data Pipeline for Robotics Physical Safety
Alexei Odinokov and Rostislav Yavorskiy
This report presents a structured Robotics Physical Safety Framework based on explicit asset declaration, systematic vulnerability enumeration, and hazard-driven synthetic data generation. The approach bridges classical risk engineering with modern machine learning pipelines, enabling safety envelope learning grounded in a formalized hazard ontology. The key contribution of this framework is the
alignment between classical safety engineering, digital twin simulation, synthetic data generation, and machine learning model training.
Read the full abstract ▾
Robotics
Glance-Say: Multimodal Human-Robot Collaboration and Intent Recognition via Sticky Glance
Yuzhi Lai, Shenghai Yuan, Peizheng Li et al. (+2 more)
Gaze and
speech are promising interaction modalities for individuals with motor impairments, yet robust intent recognition in multi-object environments remains challenging due to micro-saccades, semantic ambiguity, and viewpoint changes. This paper presents a
multimodal interaction framework for assistive robotic manipulation. We propose a sticky-glance
algorithm that stabilizes gaze-based intent by jointly accumulating
geometric distance and directional evidence, enabling robust real-time target selection and switching. We further introduce Glance-Say, a gaze-
speech interaction paradigm in which gaze specifies objects and
speech specifies actions, together with a continuous shared-control scheme that provides high-readiness
robot motion and human-in-the-loop feedback. Experiments demonstrate a tracking rate of 0.92 for moving targets, selection accuracy of 0.97 for static targets, and reduced task duration. These results indicate improved robustness, efficiency, and usability over representative interaction paradigms.
Read the full abstract ▾
All authors Yuzhi Lai, Shenghai Yuan, Peizheng Li, Benjamin Kiefer, Andreas Zell
Robotics
Multimodal Behavior Tree Generation: A Small Vision-Language Model for Robot Task Planning
Riccardo Andrea Izzo, Cristiano Battistini, Gianluca Bardaro et al. (+1 more)
Large
language models have been widely used for robotic task planning, often taking advantage of representations such as Behavior Trees (BTs).
Vision-
Language Models (VLMs) have extended these works by grounding the generated plans in the observed scene. However, existing methods are either text-only or rely on large proprietary VLMs, while no
dataset pairs visual observations and task instructions with executable and ROS2-compatible BTs. We address this gap with a multi-stage teacher pipeline that converts 1,622 Open X-Embodiment episodes into an augmented
multimodal BT
dataset containing 2,433 examples. We use this
dataset to fine-tune compact and open-source VLMs, ranging from 500M to 4B parameters, using parameter-efficient fine-tuning (PEFT). We then evaluate the generated BTs offline in terms of syntactic correctness and by executing them on 15 household tasks in BEHAVIOR-1K. Our best model, Gemma-3 4B, achieves perfect BT validity and an 87% success rate, outperforming Claude Opus 4.8 and approaching GPT-5, while running locally. Finally, our ablation studies show that adding visual observations increases task success from 40% to 87%, while data augmentation increases BT validity from 65% to 100%.
Read the full abstract ▾
All authors Riccardo Andrea Izzo, Cristiano Battistini, Gianluca Bardaro, Matteo Matteucci
Robotics
Xudong Wang, Jiahua Dong, Baichen Liu et al. (+3 more)
Embodied navigation agents powered by large
language models have shown strong performance on individual tasks but struggle to continually acquire new
navigation skills, which suffer from catastrophic forgetting. We formalize this challenge as lifelong
embodied navigation learning (LENL), where an
agent is required to adapt to a sequence of
navigation tasks spanning multiple scenes and diverse user instruction styles, while retaining previously learned knowledge. To tackle this problem, we propose Uni-Walker, a lifelong
embodied navigation framework that decouples
navigation knowledge into task-shared and task-specific components with Decoder Extension LoRA (DE-LoRA). To learn the shared knowledge, we design a knowledge inheritance strategy and an experts co-activation strategy to facilitate shared knowledge transfer and refinement across multiple
navigation tasks. To learn the specific knowledge, we propose an expert subspace orthogonality constraint together and a
navigation-specific chain-of-thought
reasoning mechanism to capture specific knowledge and enhance instruction-style understanding. Extensive experiments demonstrate the superiority of Uni-Walker for building universal
navigation agents with lifelong learning.
Read the full abstract ▾
All authors Xudong Wang, Jiahua Dong, Baichen Liu, Qi Lyu, Lianqing Liu, Zhi Han
Robotics · Cross-listed from cs.CV
Transforming Omnidirectional RGB-LiDAR data into 3D Gaussian Splatting
Semin Bae, Hansol Lim and Jongseong Brad Choi
The demand for large-scale digital twins is rapidly growing in robotics and autonomous driving. However, constructing these environments with 3D Gaussian Splatting (3DGS) usually requires expensive, purpose-built data collection. Meanwhile, deployed platforms routinely collect extensive omnidirectional RGB and
LiDAR logs, but a significant portion of these sensor data is directly discarded or strictly underutilized due to transmission constraints and the lack of scalable reuse pipeline. In this paper, we present an omnidirectional RGB-
LiDAR reuse pipeline that transforms these archived logs into robust initialization assets for 3DGS. Direct conversion of such raw logs introduces practical bottlenecks: inherent non-linear distortion leads to unreliable Structure-from-Motion (SfM) tracking, and dense, unorganized
LiDAR clouds cause computational overhead during 3DGS optimization. To overcome these challenges, our pipeline strategically integrates an ERP-to-cubemap conversion module for deterministic spatial anchoring, alongside PRISM-a color stratified downsampling strategy. By bridging these multi-modal inputs via Fast Point Feature Histograms (FPFH) based global registration and Iterative Closest Point (ICP), our pipeline successfully repurposes a considerable fraction of discarded data into usable SfM geometry. Furthermore, our
LiDAR-reinforced initialization consistently enhances the final 3DGS rendering fidelity in structurally complex scenes compared to
vision-only baselines. Ultimately, this work provides a deterministic workflow for creating simulation-grade digital twins from standard archived sensor logs.
Read the full abstract ▾
Robotics
RODEO: RObotic DEcentralized Organization
Milan Groshev and Eduardo Castelló Ferrer
Robots are improving their autonomy with minimal human supervision. However, auditable actions, transparent decision processes, and new human-
robot interaction models are still missing requirements to achieve extended
robot autonomy. To tackle these challenges, we propose RODEO (RObotic DEcentralized Organization), a blockchain-based framework that integrates trust and accountability mechanisms for
robots. This paper formalizes Decentralized Autonomous Organizations (DAOs) for service
robots. First, it provides a ROS-ETH bridge between the DAO and the
robots. Second, it offers templates that enable organizations (e.g., companies, universities) to integrate service
robots into their operations. Third, it provides proof-verification mechanisms that allow
robot actions to be auditable. In our experimental setup, a mobile
robot was deployed as a trash collector in a lab scenario. The
robot collects trash and uses a smart bin to sort and dispose of it correctly. Then, the
robot submits a proof of the successful operation and is compensated in DAO
tokens. Finally, the
robot re-invests the acquired funds to purchase battery charging services. Data collected in a three day experiment show that the
robot doubled its income and reinvested funds to extend its operating time. The proof validation times of approximately one minute ensured verifiable task execution, while the accumulated
robot income successfully funded up to 88 hours of future autonomous operation. The results of this research give insights about how
robots and organizations can coordinate tasks and payments with auditable execution proofs and on-chain settlement.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Devil is in Narrow Policy: Unleashing Exploration in Driving VLA Models
Canyu Chen, Yuguang Yang, Zhewen Tan et al. (+10 more)
We identify a fundamental Narrow Policy limitation undermining the performance of autonomous VLA models, where driving Imitation Learning (IL) tends to collapse exploration and limit the potential of subsequent Reinforcement Learning (RL) stages, which often saturate prematurely due to insufficient feedback diversity. Thereby, we propose Curious-VLA, a framework that alleviates the exploit-explore dilemma through a two-stage design. During IL, we introduce a Feasible Trajectory Expansion (FTE) strategy to generate multiple physically valid trajectories and a step-wise normalized trajectory representation to adapt this diverse data. In the RL stage, we present Adaptive Diversity-Aware Sampling (ADAS) that prioritizes high-diversity samples and introduce Spanning Driving Reward (SDR) with a focal style weighting to amplify reward's value span for improving sensitivity to driving quality. On the Navsim
benchmark, Curious-VLA achieves SoTA results (PDMS 90.3, EPDMS 85.4) and a Best-of-N PDMS of 94.8, demonstrating its effectiveness in unlocking the exploratory potential of VLA models. Code: https://github.com/Mashiroln/curious_vla.git.
Read the full abstract ▾
All authors Canyu Chen, Yuguang Yang, Zhewen Tan, Yizhi Wang, Ruiyi Zhan, Haiyan Liu, Xuanyao Mao, Jason Bao, Xinyue Tang, Linlin Yang, Bingchuan Sun, Yan Wang, Baochang Zhang
Robotics
Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration
Ninghao Zhang, Bin Zhu, Shijie Zhou et al. (+1 more)
Vision-
Language-Action (VLA) models enable
robots to perform manipulation tasks directly from natural
language instructions and are increasingly viewed as a foundation for generalist robotic policies. However, their reliability under Out-of-Distribution (OOD) instructions remains underexplored. In this paper, we reveal a critical failure mode in which VLA policies continue executing visually plausible actions even when the
language instruction contradicts the scene. We refer to this phenomenon as linguistic blindness, where VLA policies prioritize visual priors over instruction semantics during action generation. To systematically analyze this issue, we introduce ICBench, a diagnostic
benchmark constructed from the LIBERO
dataset that probes
language-action coupling by injecting controlled OOD instruction contradictions while keeping the visual environment unchanged. Evaluations on three representative VLA architectures, including Pi0, Pi0.5 and OpenVLA OFT, show that these models frequently succeed at tasks despite logically impossible instructions, revealing a strong visual bias in action generation. To mitigate this issue, we propose Instruction-Guided
Attention Recalibration (IGAR), a train-free inference-time mechanism that rebalances
attention distributions to restore the influence of
language instructions. IGAR operates without retraining or architectural modification and can be directly applied to existing VLA models. Experiments across 30 LIBERO tasks demonstrate that IGAR substantially reduces erroneous execution under OOD contradictory instructions while preserving baseline task performance. We additionally validate the approach on a real Franka robotic arm, where IGAR effectively prevents manipulation triggered by inconsistent instructions.
Read the full abstract ▾
All authors Ninghao Zhang, Bin Zhu, Shijie Zhou, Jingjing Chen
Robotics
TADPO: Reinforcement Learning Goes Off-road
Zhouchonghao Wu, Raymond Song, Vedant Mundheda et al. (+3 more)
Off-road autonomous driving poses significant challenges such as navigating unmapped, variable terrain with uncertain and diverse dynamics. Addressing these challenges requires effective long-horizon planning and adaptable control. Reinforcement Learning (RL) offers a promising solution by learning control policies directly from interaction. However, because off-road driving is a long-horizon task with low-signal rewards, standard RL methods are challenging to apply in this setting. We introduce TADPO, a novel policy gradient formulation that extends Proximal Policy Optimization (PPO), leveraging off-policy trajectories for teacher guidance and on-policy trajectories for student exploration. Building on this, we develop a
vision-based, end-to-end RL system for high-speed off-road driving, capable of navigating extreme slopes and obstacle-rich terrain. We demonstrate our performance in simulation and, importantly, zero-shot sim-to-real transfer on a full-scale off-road vehicle. To our knowledge, this work represents the first deployment of RL-based policies on a full-scale off-road platform.
Read the full abstract ▾
All authors Zhouchonghao Wu, Raymond Song, Vedant Mundheda, Luis E. Navarro-Serment, Christof Schoenborn, Jeff Schneider
Robotics
Moving Through Clutter: Scaling Data Collection and Benchmarking for 3D Scene-Aware Humanoid Locomotion via Virtual Reality
Beichen Wang, Yuanjie Lu, Linji Wang et al. (+2 more)
Recent advances in
humanoid locomotion have enabled dynamic behaviors such as dancing, martial arts, and parkour, yet these capabilities are predominantly demonstrated in open, flat, and obstacle-free settings. In contrast, real-world environments such as homes, offices, and public spaces, are densely cluttered, three-dimensional, and geometrically constrained, requiring scene-aware whole-body coordination, precise balance control, and
reasoning over spatial constraints imposed by furniture and household objects. However,
humanoid locomotion in cluttered 3D environments remains underexplored, and no public
dataset systematically couples full-body human locomotion with the scene geometry that shapes it. To address this gap, we present Moving Through Clutter (MTC), an opensource Virtual Reality (VR) based data collection and evaluation framework for scene-aware
humanoid locomotion in cluttered environments. Our system procedurally generates scenes with controllable clutter levels and captures embodiment-consistent, whole-body human motion through immersive VR
navigation, which is then automatically retargeted to a
humanoid robot model. We further introduce
benchmarks that quantify environment clutter level and locomotion performance, including stability and collision safety. Using this framework, we compile a
dataset of 348 trajectories across 145 diverse 3D cluttered scenes. The
dataset provides a foundation for studying geometry-induced adaptation in
humanoid locomotion and developing scene-aware planning and control methods.
Read the full abstract ▾
All authors Beichen Wang, Yuanjie Lu, Linji Wang, Liuchuan Yu, Xuesu Xiao
Robotics
MagRobot:An Open Simulator for Magnetically Navigated Robots
Heng Wang, Haoyu Song, Jiatao Zheng et al. (+2 more)
Magnetic navigation systems, including
magnetic tracking systems and
magnetic actuation systems, have shown great potential for occlusion-free localization and remote control of intracorporeal medical devices and
robots in minimally invasive medicine, such as capsule endoscopy and cardiovascular intervention. However, the design of magnetically navigated
robots remains heavily reliant on experimental prototyping, which is time-consuming and costly. Furthermore, there is a lack of a consistent experimental environment to compare and
benchmark the hardware and
algorithms across different
magnetic navigation systems. To address these challenges, we propose the first universal open-source simulation platform to facilitate research, design and benchmarking of magnetically navigated
robots. Our simulator features an intuitive graphical user interface that enables the user to efficiently design, visualize, and analyze
magnetic navigation systems for both rigid and soft
robots. The proposed simulator is versatile, which can simulate both
magnetic actuation and
magnetic tracking tasks in diverse medical applications that involve deformable anatomies. The proposed simulator provides an open development environment, where the user can load third-party anatomical models and customize both hardware and
algorithms of
magnetic navigation systems. The fidelity of the simulator is validated using both phantom and ex vivo experiments of
magnetic navigation of a continuum
robot and a capsule
robot with diverse
magnetic actuation setups. Three use cases of the simulator, i.e., bronchoscopy, endovascular intervention, and gastrointestinal endoscopy, are implemented to demonstrate the functionality of the simulator. It is shown that the configuration and
algorithms of
magnetic navigation systems can be flexibly designed and optimized for better performance using the simulator.
Read the full abstract ▾
All authors Heng Wang, Haoyu Song, Jiatao Zheng, Yuxiang Han, Kunli Wang
Robotics
HarvestFlex: Strawberry Harvesting via Vision-Language-Action Policy Adaptation in the Wild
Ziyang Zhao, Shuheng Wang, Zhonghua Miao et al. (+1 more)
This work presents the first study on transferring
vision-
language-action (VLA) policies to real greenhouse tabletop strawberry harvesting, a long-horizon, unstructured task challenged by occlusion and specular reflections. We built an end-to-end closed-loop system on the HarvestFlex platform using three-view RGB sensing (two fixed scene views plus a wrist-mounted view) and intentionally avoided depth clouds and explicit
geometric calibration. We collected 3.71 h of VR teleoperated demonstrations (227 episodes) and fine-tuned pi_0, pi_0.5, and WALL-OSS with full fine-tuning and LoRA. Under a unified 50 trials real-greenhouse protocol and metrics spanning completion, pi_0.5 with full fine-tuning achieved success rate of 74.0% with 32.6 s/pick and damage rate of 4.1%. Asynchronous inference-control decoupling further improved performance over synchronous deployment. Results showed non-trivial closed-loop picking with fewer than four hours of real data, while remaining limited by close-range observability loss and contact-dynamics mismatch. A demonstration
video is available at: https://youtu.be/bN8ZowZKPMI.
Read the full abstract ▾
All authors Ziyang Zhao, Shuheng Wang, Zhonghua Miao, Ya Xiong
Robotics
Proprioceptive Shape Estimation of Tensegrity Manipulators Using Energy Minimisation
Tufail Ahmad Bhat and Shuhei Ikemoto
Shape estimation is fundamental for controlling continuously bending tensegrity manipulators, yet achieving it remains a challenge. Although using exteroceptive sensors makes the implementation straightforward, it is costly and limited to specific environments. Proprioceptive approaches, by contrast, do not suffer from these limitations. So far, several methods have been proposed; however, to our knowledge, there are no proven examples of large-scale tensegrity structures used as manipulators. This paper demonstrates that shape estimation of the entire tensegrity manipulator can be achieved using only the inclination angle information relative to
gravity for each strut. Inclination angle information is intrinsic sensory data that can be obtained simply by attaching an inertial measurement unit (IMU) to each strut. Experiments conducted on a five-layer tensegrity manipulator with 20 struts and a total length of 1160 mm demonstrate that the proposed method can estimate the shape with an accuracy of 2.1 % of the total manipulator length, from arbitrary initial conditions under both static conditions and maintains stable shape estimation under external disturbances.
Read the full abstract ▾
Robotics
PROBE: Probabilistic Occupancy BEV Encoding with Analytical Translation Robustness for 3D Place Recognition
Jinseop Lee, Byoungho Lee and Gichul Yoo
We present PROBE (PRobabilistic Occupancy BEV Encoding), a learning-free
LiDAR place recognition descriptor that models each BEV
cell's occupancy as a Bernoulli random variable. Rather than relying on discrete point-cloud perturbations, PROBE analytically marginalizes over continuous Cartesian translations via the polar Jacobian, yielding a distance-adaptive angular uncertainty $\sigma_\theta = \sigma_t / r$ in $\mathcal{O}(R{\cdot}S)$ time. The primary parameter $\sigma_t$ represents the expected translational uncertainty in meters, a sensor-independent physical quantity that enhances cross-sensor generalization while reducing the need for extensive per-
dataset tuning. Pairwise similarity combines a Bernoulli-KL Jaccard with exponential uncertainty gating and FFT-based height cosine similarity for rotation
alignment. Evaluated on four
datasets spanning four diverse
LiDAR types, PROBE achieves the highest accuracy among handcrafted descriptors in multi-session evaluation and competitive single-session performance relative to both handcrafted and supervised baselines. The source code and supplementary materials are available at https://sites.google.com/view/probe-pr.
Read the full abstract ▾
Robotics
How to Model Your Crazyflie Brushless
Alexander Gräfe, Christoph Scherer, Wolfgang Hönig et al. (+1 more)
The Crazyflie quadcopter is widely recognized as a leading platform for nano-quadcopter research. In early 2025, the Crazyflie Brushless was introduced, featuring brushless motors that provide around 50% more thrust compared to the brushed motors of its predecessor, the Crazyflie 2.1. This advancement has opened new opportunities for research in agile nano-quadcopter control. To support researchers utilizing this new platform, this work presents a dynamics model of the Crazyflie Brushless and identifies its key parameters. Through simulations and hardware analyses, we assess the accuracy of our model. We furthermore demonstrate its suitability for reinforcement learning applications by training an end-to-end neural network position controller and learning a backflip controller capable of executing two complete rotations with a vertical movement of just 1.8 meters. This showcases the model's ability to facilitate the learning of controllers and acrobatic maneuvers that successfully transfer from simulation to hardware. Utilizing this application, we investigate the impact of domain randomization on control performance, offering valuable insights into bridging the sim-to-real gap with the presented model. We have open-sourced the entire project, enabling users of the Crazyflie Brushless to swiftly implement and test their own controllers on an accurate simulation platform.
Read the full abstract ▾
All authors Alexander Gräfe, Christoph Scherer, Wolfgang Hönig, Sebastian Trimpe
Robotics
Swooper: Learning High-Speed Aerial Grasping With a Simple Gripper
Ziken Huang, Xinze Niu, Bowen Chai et al. (+2 more)
High-speed aerial grasping presents significant challenges due to the high demands on precise, responsive flight control and coordinated gripper manipulation. In this work, we propose Swooper, a deep reinforcement learning (DRL) based approach that achieves both precise flight control and active gripper control using a single lightweight neural network policy. Training such a policy directly via DRL is nontrivial due to the complexity of coordinating flight and grasping. To address this, we adopt a two-stage learning strategy: we first pre-train a flight control policy, and then fine-tune it to acquire grasping skills. With the carefully designed reward functions and training framework, the entire training process completes in under 60 minutes on a standard desktop with an Nvidia RTX 3060 GPU. To validate the trained policy in the real world, we develop a lightweight quadrotor grasping platform equipped with a simple off-the-shelf gripper, and deploy the policy in a zero-shot manner on the onboard Raspberry Pi 4B computer, where each inference takes only about 1.0 ms. In 25 real-world trials, our policy achieves an 84% grasp success rate and grasping speeds of up to 1.5 m/s without any fine-tuning. This matches the robustness and agility of state-of-the-art classical systems with sophisticated grippers, highlighting the capability of DRL for learning a robust control policy that seamlessly integrates high-speed flight and grasping. The supplementary
video is available for more results.
Video: https://zikenhuang.github.io/Swooper/.
Read the full abstract ▾
All authors Ziken Huang, Xinze Niu, Bowen Chai, Renbiao Jin, Danping Zou
Robotics · Cross-listed from cs.CV
FTSplat: Feed-forward Triangle Splatting Network
Xiong Jinlin, Li Can, Shen Jiawei et al. (+3 more)
High-fidelity three-dimensional (3D) reconstruction is essential for robotics and simulation. While Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) achieve impressive rendering quality, their reliance on time-consuming per-scene optimization limits real-time deployment. Emerging feed-forward Gaussian splatting methods improve efficiency but often lack explicit,
manifold geometry required for direct simulation. To address these limitations, we propose a feed-forward framework for triangle primitive generation that directly predicts continuous triangle surfaces from calibrated multi-view images. Our method produces simulation-ready models in a single forward pass, obviating the need for per-scene optimization or post-processing. We introduce a pixel-aligned triangle generation module and incorporate relative 3D point cloud supervision to enhance
geometric learning stability and consistency. Experiments demonstrate that our method achieves efficient reconstruction while maintaining seamless compatibility with standard graphics and robotic simulators.
Read the full abstract ▾
All authors Xiong Jinlin, Li Can, Shen Jiawei, Qi Zhigang, Sun Lei, Zhao Dongyang
Robotics
Iterative Convex Optimization with Control Barrier Functions for Obstacle Avoidance among Polytopes
Shuo Liu, Zhe Huang and Calin A. Belta
Obstacle avoidance of polytopic obstacles by polytopic
robots is a challenging problem in optimization-based control and trajectory planning. Many existing methods rely on smooth
geometric approximations, such as hyperspheres or ellipsoids, which allow differentiable distance expressions but distort the true geometry and restrict the feasible set. Other approaches integrate exact polytope distances into nonlinear model predictive control (MPC), resulting in nonconvex programs that limit real-time performance. In this paper, we construct linear discrete-time control barrier function (DCBF) constraints by deriving supporting hyperplanes from exact closest-point computations between convex polytopes. We then propose a novel iterative convex MPC-DCBF framework, where local linearization of system dynamics and
robot geometry ensures convexity of the finite-horizon optimization at each iteration. The resulting formulation reduces computational complexity and enables fast online implementation for safety-critical control and trajectory planning of general nonlinear dynamics. The framework extends to multi-
robot and three-dimensional environments. Numerical experiments demonstrate collision-free
navigation in cluttered maze scenarios with millisecond-level solve times.
Read the full abstract ▾
Robotics
Improved hopping control on slopes for small robots using spring mass modeling
Heston Roberts, Pronoy Sarker, Sm Ashikul Islam et al. (+1 more)
Hopping
robots often lose balance on slopes because the tilted ground creates unwanted rotation at landing. This work analyzes that effect using a simple spring mass model and identifies how slope induced impulses destabilize the
robot. To address this, we introduce two straightforward fixes, adjusting the bodys touchdown angle based on the slope and applying a small corrective torque before takeoff. Together, these steps effectively cancel the unwanted rotation caused by inclined terrain, allowing the
robot to land smoothly and maintain stable hopping even on steep slopes. Moreover, the proposed method remains simple enough to implement on low cost robotic platforms without requiring complex sensing or computation. By combining this analytical model with minimal control actions, this approach provides a practical path toward reliable hopping on uneven terrain. The results from simulation confirm that even small slope aware adjustments can dramatically improve landing stability, making the technique suitable for future autonomous field
robots that must navigate natural environments such as hills, rubble, and irregular outdoor landscapes.
Read the full abstract ▾
All authors Heston Roberts, Pronoy Sarker, Sm Ashikul Islam, Min Gyu Kim
Robotics · Cross-listed from cs.CV
Systematic Evaluation of Novel View Synthesis for Video Place Recognition
Muhammad Zawad Mahmud, Samiha Islam and Damian Lyons
The generation of synthetic novel views has the potential to positively impact
robot navigation in several ways. In image-based
navigation, a novel overhead view generated from a scene taken by a ground
robot could be used to guide an aerial
robot to that location. In
Video Place Recognition (VPR), novel views of ground locations from the air can be added that enable a
UAV to identify places seen by the ground
robot, and similarly, overhead views can be used to generate novel ground views. This paper presents a systematic evaluation of synthetic novel views in VPR using five public VPR image databases and seven typical image similarity methods. We show that for small synthetic additions, novel views improve VPR recognition statistics. We find that for larger additions, the magnitude of viewpoint change is less important than the number of views added and the type of imagery in the
dataset.
Read the full abstract ▾
Robotics
AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models
Hyeongjun Heo, Seungyeon Woo, Sang Min Kim et al. (+4 more)
Despite remarkable progress in
Vision-
Language-Action models (VLAs) for
robot manipulation, these large pre-trained models require fine-tuning to be deployed in specific environments. These fine-tuned models are highly sensitive to camera viewpoint changes that frequently occur in unstructured environments. In this paper, we propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification. Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time. For that, we use a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. This plug-and-play approach preserves the pre-trained capabilities of VLAs and applies to any RGB-based policy. Through extensive experiments on the LIBERO
benchmark, our method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. We further validate that our approach constantly enhances viewpoint robustness in real-world robotic manipulation scenarios, including settings with varying camera extrinsics, intrinsics, and freely moving handheld cameras. Project Page: https://heo0224.github.io/AnyCamVLA/
Read the full abstract ▾
All authors Hyeongjun Heo, Seungyeon Woo, Sang Min Kim, Junho Kim, Junho Lee, Yonghyeon Lee, Young Min Kim
Robotics
DexEMG: Towards Dexterous Teleoperation System via EMG2Pose Generalization
Qianyou Zhao, Wenqiao Li, Chiyu Wang et al. (+1 more)
High-fidelity teleoperation of dexterous robotic hands is essential for bringing
robots into unstructured domestic environments. However, existing teleoperation systems often face a trade-off between performance and portability:
vision-based capture systems are constrained by costs and line-of-sight requirements, while mechanical exoskeletons are bulky and physically restrictive. In this paper, we present DexEMG, a lightweight and cost-effective teleoperation system leveraging surface electromyography (sEMG) to bridge the gap between human intent and robotic execution. We first collect a synchronized
dataset of sEMG signals and hand poses via a MoCap glove to train EMG2Pose, a neural network capable of continuously predicting hand kinematics directly from muscle activity. To ensure seamless control, we develop a robust hand retargeting
algorithm that maps the predicted poses onto a multi-fingered dexterous hand in real-time. Experimental results demonstrate that DexEMG achieves high precision in diverse teleoperation tasks. Notably, our system exhibits strong generalization capabilities across novel objects and complex environments without the need for intensive individual-specific recalibration. This work offers a scalable and intuitive interface for both general-purpose robotic manipulation and assistive technologies.
Read the full abstract ▾
All authors Qianyou Zhao, Wenqiao Li, Chiyu Wang, Kaifeng Zhang
Robotics
Expert Knowledge-driven Reinforcement Learning for Autonomous Racing via Trajectory Guidance and Dynamics Constraints
Bo Leng, Weiqi Zhang, Zhuoren Li et al. (+4 more)
Reinforcement learning has shown significant potential for autonomous racing, but it still faces challenges such as training instability, inefficient exploration, and unsafe action outputs in high-dynamic racing scenarios. This paper proposes a Trajectory guidance and Dynamics constraints Reinforcement Learning (TraD-RL) framework for autonomous racing. The proposed method incorporates expert prior knowledge into policy learning through Minimum
Curvature Racing Line (MCRL) guidance, explicit vehicle dynamics constraints, and two-stage curriculum learning. MCRL provides global path and velocity references through observation augmentation and reward shaping, thereby improving exploration efficiency and racing performance. Yaw rate and sideslip angle constraints are introduced to characterize the vehicle dynamic safe operating envelope, and the corresponding stability costs are incorporated into policy optimization through Lagrangian relaxation. Moreover, the two-stage curriculum learning strategy enables a progressive transition from stable trajectory following to high-speed performance exploration. Experiments on two racetracks demonstrate that TraD-RL improves racing performance while maintaining a favorable balance between speed and dynamic stability. Further analyses of ablation, sensitivity, and robustness validate the effectiveness and stability of the proposed framework.
Read the full abstract ▾
All authors Bo Leng, Weiqi Zhang, Zhuoren Li, Lu Xiong, Guizhe Jin, Ran Yu, Chen Lv
Robotics
Terrain characterization and locomotion adaptation in a small-scale lizard-inspired robot
Duncan Andrews, Landon Zimmerman, Evan Martin et al. (+2 more)
Unlike their large-scale counterparts, small-scale
robots are largely confined to laboratory environments and are rarely deployed in real-world settings. As
robot size decreases,
robot-terrain interactions fundamentally change; however, there remains a lack of systematic understanding of what sensory information small-scale
robots should acquire and how they should respond when traversing complex natural terrains. To address these challenges, we develop a Small-scale, Intelligent, Lizard-inspired, Adaptive
Robot (SILA Bot) capable of adapting to diverse substrates. We use granular media of varying depths as a controlled yet representative terrain paradigm. We show that the optimal body movement pattern (ranging from standing-wave bending that assists limb retraction on flat ground to traveling-wave undulation that generates thrust in deep granular media) can be parameterized and approximated as a linear function of granular depth. Furthermore, proprioceptive signals, such as joint torque, provide sufficient information to estimate granular depth via a K-Nearest Neighbors classifier, achieving 95% accuracy. Leveraging these relationships, we design a simple linear feedback controller that modulates body phase and substantially improves locomotion performance on terrains with unknown depth. Together, these results establish a principled framework for perception and control in small-scale locomotion and enable effective terrain-adaptive locomotion while maintaining low computational complexity.
Read the full abstract ▾
All authors Duncan Andrews, Landon Zimmerman, Evan Martin, Joe DiGennaro, Baxi Chong
Robotics
OpenHEART: Opening Heterogeneous Articulated Objects with a Legged Manipulator
Seonghyeon Lim, Hyeonwoo Lee, Seunghyun Lee et al. (+2 more)
Legged manipulators offer high mobility and versatile manipulation. However, robust interaction with heterogeneous articulated objects, such as doors, drawers, and cabinets, remains challenging because of the diverse articulation types of the objects and the complex dynamics of the legged
robot. Existing reinforcement learning (RL)-based approaches often rely on high-dimensional sensory inputs, leading to sample inefficiency. In this paper, we propose a robust and sample-efficient framework for opening heterogeneous articulated objects with a legged manipulator. In particular, we propose Sampling-based Abstracted Feature Extraction (SAFE), which encodes handle and panel geometry into a compact low-dimensional representation, improving cross-domain generalization. Additionally, Articulation Information
Estimator (ArtIEst) is introduced to adaptively mix proprioception with exteroception to estimate opening direction and range of motion for each object. The proposed framework was deployed to manipulate various heterogeneous articulated objects in simulation and real-world
robot systems.
Videos can be found on the project website: https://openheart-icra.github.io/OpenHEART/
Read the full abstract ▾
All authors Seonghyeon Lim, Hyeonwoo Lee, Seunghyun Lee, I Made Aswin Nahrendra, Hyun Myung
Robotics
Hierarchical Latent Action Model
Hanjung Kim, Lerrel Pinto and Seon Joo Kim
Latent Action Models (LAMs) enable learning from actionless data for applications ranging from robotic control to interactive world models. However, existing LAMs typically focus on short-horizon frame transitions and capture low-level motion while overlooking longer-term temporal structure. In contrast, actionless
videos often contain temporally extended and high-level skills. We present HiLAM, a hierarchical latent action model that discovers latent skills by modeling long-term temporal information. To capture these dependencies across long horizons, we utilize a pretrained LAM as a low-level extractor. This architecture aggregates latent action sequences, which contain the underlying dynamic patterns of the
video, into high-level latent skills. Our experiments demonstrate that HiLAM improves over the baseline and exhibits robust dynamic skill discovery.
Read the full abstract ▾
Robotics
CDF-Glove: A Cable-Driven Force Feedback Glove for Dexterous Teleoperation
Huayue Liang, Ruochong Li, Yaodong Yang et al. (+3 more)
High-quality teleoperated demonstrations are a primary bottleneck for imitation learning (IL) in dexterous manipulation. However, haptic feedback provides operators with real-time contact information, enabling real-time finger posture adjustments, and thereby improving demonstration quality. Existing dexterous teleoperation platforms typically omit haptic feedback and remain bulky and expensive. We introduce CDF-Glove, a lightweight and low cost cable-driven force-feedback glove. The real-time state is available for 20 finger degrees of freedom (DoF), of which 16 are directly sensed and 4 are passively coupled (inferred from kinematic constraints). We develop a kinematic model and control stack for the glove, and validate them across multiple robotic hands with diverse kinematics and DoF. The CDF-Glove achieves distal joint repeatability of 0.4 degrees, and delivers about 200 ms force feedback latency, yielding a 4x improvement in task success rate relative to no-feedback teleoperation. We collect two bimanual teleoperation
datasets, on which we train and evaluate
Diffusion Policy baselines. Compared to kinesthetic teaching, the policies trained in our teleoperated demonstrations increase the average success rate by 55% and reduce the mean completion time by approximately 15.2 seconds (a 47.2% relative reduction). In particular, the CDF-Glove costs approximately US$230. The code and designs are released as open source at https://cdfglove.github.io/.
Read the full abstract ▾
All authors Huayue Liang, Ruochong Li, Yaodong Yang, Long Zeng, Yuanpei Chen, Xueqian Wang
Robotics
Task-Level Decisions to Gait Level Control: A Hierarchical Policy Approach for Quadruped Navigation
Sijia Li, Haoyu Wang, Shenghai Yuan et al. (+2 more)
Real-world quadruped
navigation is constrained by a scale mismatch between high-level
navigation decisions and low-level gait execution, as well as by instabilities under out-of-distribution environmental changes. Such variations challenge sim-to-real transfer and can trigger falls when policies lack explicit interfaces for adaptation. In this paper, we present a hierarchical policy architecture for quadrupedal
navigation, termed Task-level Decision to Gait Control (TDGC). A low-level policy, trained with reinforcement learning in simulation, delivers gait-conditioned locomotion and maps task requirements to a compact set of controllable behavior parameters, enabling robust mode generation and smooth switching. A high-level policy makes task-centric decisions from sparse semantic or
geometric terrain cues and translates them into low-level targets, forming a traceable decision pipeline without dense maps or high-resolution terrain reconstruction. Different from end-to-end approaches, our architecture provides explicit interfaces for deployment-time tuning, fault diagnosis, and policy refinement. We introduce a structured curriculum with performance-driven progression that expands environmental difficulty and disturbance ranges. Experiments show higher task success rates on mixed terrains and out-of-distribution tests.
Read the full abstract ▾
All authors Sijia Li, Haoyu Wang, Shenghai Yuan, Yizhuo Yang, Thien-Minh Nguyen
Robotics
Multi-Robot Trajectory Planning via Constrained Bayesian Optimization and Local Cost Map Learning with STL-Based Conflict Resolution
Sourav Raxit, Abdullah Al Redwan Newaz, Jose Fuentes et al. (+3 more)
We address multi-
robot motion planning under Signal Temporal Logic (STL) specifications with kinodynamic constraints. Exact approaches face scalability bottlenecks and limited adaptability, while conventional sampling-based methods require excessive samples to construct optimal trajectories. We propose a two-stage framework integrating sampling-based online learning with formal STL
reasoning. At the single-
robot level, our constrained
Bayesian Optimization-based Tree search (cBOT) planner uses a Gaussian process as a surrogate model to learn local cost maps and feasibility constraints, generating shorter collision-free trajectories with fewer samples. At the multi-
robot level, our STL-enhanced Kinodynamic Conflict-Based Search (STL-KCBS)
algorithm incorporates STL monitoring into conflict detection and resolution, ensuring specification satisfaction while maintaining scalability and probabilistic completeness. Benchmarking demonstrates improved trajectory efficiency and safety over existing methods. Real-world experiments with autonomous surface vehicles validate robustness and practical applicability in uncertain environments. The STLcBOT Planner will be released as an open-source package, and
videos of real-world and simulated experiments are available at https://stlbot.github.io/.
Read the full abstract ▾
All authors Sourav Raxit, Abdullah Al Redwan Newaz, Jose Fuentes, Paulo Padrao, Ana Cavalcanti, Leonardo Bobadilla