Robotics
Warm-Starting Optimization-Based Motion Planning for Robotic Manipulators via Point Cloud-Conditioned Flow Matching
Sibo Tian, Minghui Zheng and Xiao Liang
Rapid
robot motion generation is critical in Human-
Robot Collaboration (HRC) systems, as
robots need to respond to dynamic environments in real time by continuously observing their surroundings and replanning their motions to ensure both safe interactions and efficient task execution. Current sampling-based motion planners face challenges in scaling to high-dimensional configuration spaces and often require post-processing to interpolate and smooth the generated paths, resulting in time inefficiency in complex environments. Optimization-based planners, on the other hand, can incorporate multiple constraints and generate smooth trajectories directly, making them potentially more time-efficient. However, optimization-based planners are sensitive to initialization and may get stuck in local minima. In this work, we present a novel learning-based method that utilizes a Flow Matching model conditioned on a single-view point cloud to learn near-optimal solutions for optimization initialization. Our method does not require prior knowledge of the environment, such as obstacle locations and geometries, and can generate feasible trajectories directly from single-view depth camera input. Simulation studies on a UR5e robotic manipulator in cluttered workspaces demonstrate that the proposed generative initializer achieves a high success rate on its own, significantly improves the success rate of trajectory optimization compared with traditional and learning-based
benchmark initializers, requires fewer optimization iterations, and exhibits strong generalization to unseen environments.
Read the full abstract ▾
Robotics
Optimal swimming with body compliance in an overdamped medium
Jianfeng Lin, Tianyu Wang, Baxi Chong et al. (+3 more)
Elongate animals and
robots use undulatory body waves to locomote through diverse environments.
Geometric mechanics provides a framework to model and optimize such systems in highly damped environments, connecting a prescribed shape change pattern (gait) with locomotion displacement. However, the practical applicability of controlling compliant physical
robots remains to be demonstrated. In this work, we develop a framework based on
geometric mechanics to predict locomotor performance and search for optimal swimming strategies of compliant swimmers. We introduce a compliant extension of Purcell's three-link swimmer by incorporating series-connected springs at the joints. Body dynamics are derived using resistive force theory.
Geometric mechanics is incorporated into movement prediction and into an optimization framework that identifies strategies for controlling compliant swimmers to achieve maximal displacement. We validate our framework on a physical cable-driven three-link limbless
robot and demonstrate accurate prediction and optimization of locomotor performance under varied programmed, state-dependent compliance in a granular medium. Our results establish a systematic, physics-based approach for modeling and controlling compliant swimming locomotion, highlighting compliance as a design feature that can be exploited for robust movement in both homogeneous and heterogeneous environments.
Read the full abstract ▾
All authors Jianfeng Lin, Tianyu Wang, Baxi Chong, Matthew Fernandez, Zhaochen Xu, Daniel I. Goldman
Robotics · Cross-listed from eess.SY
Viability-Preserving Passive Torque Control
Zizhe Zhang, Yicong Wang, Zhiquan Zhang et al. (+2 more)
Conventional passivity-based torque controllers for manipulators are typically unconstrained, which can lead to safety violations under external perturbations. In this paper, we employ viability theory to pre-compute safe sets in the state-space of joint positions and velocities. These viable sets, constructed via data-driven and analytical methods for self-collision avoidance, external object collision avoidance and joint-position and joint-velocity limits, provide constraints on joint accelerations and thus joint torques via the
robot dynamics. A quadratic programming-based control framework enforces these constraints on a passive controller tracking a dynamical system, ensuring the
robot states remain within the safe set in an infinite time horizon. We validate the proposed approach through simulations and hardware experiments on a 7-DoF Franka Emika manipulator. In comparison to a baseline constrained passive controller, our method operates at higher control-loop rates and yields smoother trajectories.
Read the full abstract ▾
All authors Zizhe Zhang, Yicong Wang, Zhiquan Zhang, Tianyu Li, Nadia Figueroa
Robotics
Simulation to Rules: A Dual-VLM Framework for Formal Visual Planning
Yilun Hao, Yongchao Chen, Chuchu Fan et al. (+1 more)
Vision Language Models (VLMs) show strong potential for visual planning but struggle with precise spatial and long-horizon
reasoning, while Planning Domain Definition
Language (PDDL) planners excel at formal long-horizon planning but cannot interpret visual inputs. Recent works combine these complementary advantages by translating visual problems into PDDL. However, while VLMs can generate PDDL problem files satisfactorily, accurately generating PDDL domain files, which encode planning rules, remains challenging and typically requires human expertise or environment interaction. We propose VLMFP, a Dual-VLM-guided framework that autonomously generates both PDDL problem and domain files for formal visual planning. VLMFP combines a SimVLM that simulates action consequences with a GenVLM that generates and iteratively refines PDDL files by aligning symbolic execution with simulated outcomes, enabling multiple levels of generalization across unseen instances, visual appearances, and game rules. We evaluate VLMFP on 6 grid-world domains and demonstrate its generalization capability. On average, SimVLM achieves 87.3% and 86.0% scenario understanding and action simulation for seen and unseen appearances, respectively. With the guidance of SimVLM, VLMFP attains 70.0%, 54.1% planning success on unseen instances in seen and unseen appearances, respectively. We further demonstrate that VLMFP scales to complex long-horizon 3D planning tasks, including multi-
robot collaboration and assembly scenarios with partial observability and diverse visual variations. Project page: https://sites.google.com/view/vlmfp.
Read the full abstract ▾
All authors Yilun Hao, Yongchao Chen, Chuchu Fan, Yang Zhang
Robotics
Optimal Smooth Coverage Trajectory Planning for Quadrotors in Cluttered Environment
Duanjiao Li, Yun Chen, Ying Zhang et al. (+4 more)
For typical applications of
UAVs in power grid scenarios, we construct the problem as planning
UAV trajectories for coverage in cluttered environments. In this paper, we propose an optimal smooth coverage trajectory planning
algorithm. The
algorithm consists of two stages. In the front-end, a Genetic
Algorithm (GA) is employed to solve the Traveling Salesman Problem (TSP) for Points of Interest (POIs), generating an initial sequence of optimized visiting points. In the back-end, the sequence is further optimized by considering trajectory smoothness, time consumption, and obstacle avoidance. This is formulated as a nonlinear least squares problem and solved to produce a smooth coverage trajectory that satisfies these constraints. Numerical simulations validate the effectiveness of the proposed
algorithm, ensuring
UAVs can smoothly cover all POIs in cluttered environments.
Read the full abstract ▾
All authors Duanjiao Li, Yun Chen, Ying Zhang, Junwen Yao, Dongyue Huang, Jianguo Zhang, Ning Ding
Robotics · Cross-listed from cs.AI
Improving Cooperation in Collaborative Embodied AI
Hima Jacob Leven Suprabha, Laxmi Nag Laxminarayan Nagesh, Ajith Nair et al. (+10 more)
The integration of Large
Language Models (LLMs) into multiagent systems has opened new possibilities for collaborative
reasoning and cooperation with AI
agents. This paper explores different prompting methods and evaluates their effectiveness in enhancing
agent collaborative behaviour and decision-making. We enhance CoELA, a framework designed for building Collaborative
Embodied Agents that leverage LLMs for multi-
agent communication,
reasoning, and task coordination in shared virtual spaces. Through systematic experimentation, we examine different LLMs and
prompt engineering strategies to identify optimised combinations that maximise collaboration performance. Furthermore, we extend our research by integrating
speech capabilities, enabling seamless collaborative voice-based interactions. Our findings highlight the effectiveness of
prompt optimisation in enhancing collaborative
agent performance; for example, our best combination improved the efficiency of the system running with Gemma3 by 22% compared to the original CoELA system. In addition, the
speech integration provides a more engaging user interface for iterative system development and demonstrations.
Read the full abstract ▾
All authors Hima Jacob Leven Suprabha, Laxmi Nag Laxminarayan Nagesh, Ajith Nair, Alvin Reuben Amal Selvaster, Ayan Khan, Raghuram Damarla, Sanju Hannah Samuel, Sreenithi Saravana Perumal, Titouan Puech, Venkataramireddy Marella, Vishal Sonar, Alessandro Suglia, Oliver Lemon
Robotics
MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learning
Tianyu Xu, Jiawei Chen, Jiazhao Zhang et al. (+5 more)
Visual
navigation policy is widely regarded as a promising direction, as it mimics humans by using
egocentric visual observations for
navigation. However, optical information of visual observations is difficult to be explicitly modeled like
LiDAR point clouds or depth maps, which subsequently requires intelligent models and large-scale data. To this end, we propose to leverage the intelligence of the
Vision-
Language-Action (VLA) model to learn diverse
navigation capabilities from synthetic expert data in a teacher-student manner. Specifically, we implement the VLA model, MM-Nav, as a multi-view VLA (with 360 observations) based on pretrained large
language models and visual foundation models. For large-scale
navigation data, we collect expert data from three reinforcement learning (RL) experts trained with privileged depth information in three challenging tailor-made environments for different
navigation capabilities: reaching, squeezing, and avoiding. We iteratively train our VLA model using data collected online from RL experts, where the training ratio is dynamically balanced based on performance on individual capabilities. Through extensive experiments in synthetic environments, we demonstrate that our model achieves strong generalization capability. Moreover, we find that our student VLA model outperforms the RL teachers, demonstrating the synergistic effect of integrating multiple capabilities. Extensive real-world experiments further confirm the effectiveness of our method.
Read the full abstract ▾
All authors Tianyu Xu, Jiawei Chen, Jiazhao Zhang, Wenyao Zhang, Zekun Qi, Minghan Li, Zhizheng Zhang, He Wang
Robotics · Cross-listed from cs.CV
Mask2IV: Interaction-Centric Video Generation via Mask Trajectories
Gen Li, Bo Zhao, Jianfei Yang et al. (+1 more)
Generating interaction-centric
videos, such as those depicting humans or
robots interacting with objects, is crucial for
embodied intelligence, as they provide rich and diverse visual priors for
robot learning, manipulation policy training, and affordance
reasoning. However, existing methods often struggle to model such complex and dynamic interactions. While recent studies show that masks can serve as effective control signals and enhance generation quality, obtaining dense and precise mask annotations remains a major challenge for real-world use. To overcome this limitation, we introduce Mask2IV, a novel framework specifically designed for interaction-centric
video generation. It adopts a decoupled two-stage pipeline that first predicts plausible motion trajectories for both actor and object, then generates a
video conditioned on these trajectories. This design eliminates the need for dense mask inputs from users while preserving the flexibility to manipulate the interaction process. Furthermore, Mask2IV supports versatile and intuitive control, allowing users to specify the target object of interaction and guide the motion trajectory through action descriptions or spatial position cues. To support systematic training and evaluation, we curate two
benchmarks covering diverse action and object categories across both human-object interaction and robotic manipulation scenarios. Extensive experiments demonstrate that our method achieves superior visual realism and controllability compared to existing baselines.
Read the full abstract ▾
All authors Gen Li, Bo Zhao, Jianfei Yang, Laura Sevilla-Lara
Robotics
Learning Stability Certificate for Robotics in Real-World Environments
Zhe Shen
Stability certificates play a critical role in ensuring the safety and reliability of robotic systems. However, deriving these certificates for complex, unknown systems has traditionally required explicit knowledge of system dynamics, often making it a daunting task. This work introduces a novel framework that learns a Lyapunov function directly from trajectory data, enabling the certification of stability for autonomous systems without needing detailed system models. By parameterizing the Lyapunov candidate using a neural network and ensuring positive definiteness through Cholesky factorization, our approach automatically identifies whether the system is stable under the given trajectory. To address the challenges posed by noisy, real-world data, we allow for controlled violations of the stability condition, focusing on maintaining high confidence in the stability certification process. Our results demonstrate that this framework can provide data-driven stability guarantees, offering a robust method for certifying the safety of robotic systems in dynamic, real-world environments. This approach works without access to the internal control
algorithms, making it applicable even in situations where system behavior is opaque or proprietary. The tool for learning the stability proof is open-sourced by this research: https://github.com/HansOersted/stability.
Read the full abstract ▾
Robotics
Whisker-based Tactile Flight for Tiny Drones
Chaoxiang Ye, Guido de Croon and Salua Hamaza
Tiny flying
robots hold great potential for search-and-rescue, safety inspections, and environmental monitoring, but their small size and limited computational resources constrain onboard sensing capabilities. Inspired by animals such as rats and moles, which rely on lightweight whiskers to navigate and perceive their surroundings through touch, we present a 3.2-gram whisker-based tactile sensing apparatus that enables tiny drones to perceive and interact with their environment through gentle physical contact, even in complete darkness. The apparatus employs barometers at the base of each whisker to estimate contact depth, enabling obstacle localization while minimizing contact-induced destabilization. To compensate for sensor noise and drift during sustained contact, we develop a tactile depth estimation pipeline that achieves millimeter-scale depth estimation accuracy. Together, these innovations enable tiny drones to autonomously avoid obstacles, contour surfaces, and explore confined spaces, guided by onboard tactile sensing across both rigid and soft environments. Running entirely onboard a microcontroller with just 192 KB of
memory, our system demonstrates autonomous tactile flight across various scenarios. This bio-inspired approach extends perception for mobile
robots beyond
vision, opening new possibilities for autonomous operations in visually degraded and GPS-denied environments.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Geometry Meets Vision: Revisiting Pretrained Semantics in Distilled Fields
Zhiting Mei, Ola Shorinwa and Anirudha Majumdar
Semantic
distillation in radiance fields has spurred significant advances in open-vocabulary
robot policies, e.g., in manipulation and
navigation, founded on pretrained semantics from large
vision models. While prior work has demonstrated the effectiveness of visual-only semantic features (e.g., DINO and CLIP) in Gaussian Splatting and neural radiance fields, the potential benefit of geometry-grounding in distilled fields remains an open question. In principle, visual-geometry features seem very promising for spatial tasks such as pose estimation, prompting the question: Do geometry-grounded semantic features offer an edge in distilled fields? Specifically, we ask three critical questions: First, does spatial-grounding produce higher-fidelity geometry-aware semantic features? We find that image features from geometry-grounded backbones contain finer structural details compared to their counterparts. Secondly, does geometry-grounding improve semantic object localization? We observe no significant difference in this task. Thirdly, does geometry-grounding enable higher-accuracy radiance field inversion? Given the limitations of prior work and their lack of semantics integration, we propose a novel framework SPINE for inverting radiance fields without an initial guess, consisting of two core components: coarse inversion using distilled semantics, and fine inversion using photometric-based optimization. Surprisingly, we find that the pose estimation accuracy decreases with geometry-grounded features. Our results suggest that visual-only features offer greater versatility for a broader range of downstream tasks, although geometry-grounded features contain more
geometric detail. Notably, our findings underscore the necessity of future research on effective strategies for geometry-grounding that augment the versatility and performance of pretrained semantic features.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
A Dimension-Decomposed Learning Framework for Online Disturbance Identification in Quadrotor SE(3) Control
Tianhua Gao
Quadrotor stability under complex dynamic disturbances and model uncertainties poses significant challenges. One of them remains the underfitting problem in high-dimensional features, which limits the identification capability of current learning-based methods. To address this, we introduce a new perspective: Dimension-Decomposed Learning (DiD-L), from which we develop the Sliced Adaptive-Neuro Mapping (SANM) approach for
geometric control. Specifically, the high-dimensional mapping for identification is axially “sliced" into multiple low-dimensional submappings (“slices"). In this way, the complex high-dimensional problem is decomposed into a set of simple low-dimensional tasks addressed by shallow neural networks and adaptive laws. These neural networks and adaptive laws are updated online via Lyapunov-based adaptation without any pre-training or persistent excitation (PE) condition. To enhance the interpretability of the proposed approach, we prove that the full-state closed-loop system exhibits arbitrarily close to exponential stability despite multi-dimensional time-varying disturbances and model uncertainties. This result is novel as it demonstrates exponential convergence without requiring pre-training for unknown disturbances and specific knowledge of the model.
Read the full abstract ▾
Robotics
Toward Physical Recursive Self-Improvement: A Call for Body-Control Co-Design for Humanoid Robot
Guiliang Liu, Bo Yue, Yi Jin Kim et al. (+1 more)
Humanoid robots, as general-purpose physical
agents, must integrate both intelligent control and adaptive morphology to operate effectively in diverse real-world environments. While recent research primarily optimizes control policies for fixed bodies, we advocate body-control co-design as a path toward physical Recursive Self-Improvement (RSI): using experience from a
humanoid's current body and policy to guide successive improvements to both. Inspired by biological
evolution, this approach enables
robots to recursively adapt both their form and behavior to optimize performance within task-specific and resource-constrained contexts. Despite its promise, co-design in
humanoid robotics remains a relatively underexplored domain, raising fundamental questions about its feasibility and necessity in achieving true
embodied intelligence. To address these challenges, we discuss potential co-design methodologies grounded in strategic exploration, Sim2Real transfer, and meta-policy learning. We further argue for the essential role of co-design by analyzing it from methodological, application-driven, and community-oriented perspectives. Striving to guide and inspire future studies, we present open research questions, spanning from short-term innovations to long-term goals. Ultimately, co-design offers a path toward physical RSI in
humanoids that improves their bodies and policies while learning to make this improvement process more effective.
Read the full abstract ▾
All authors Guiliang Liu, Bo Yue, Yi Jin Kim, Kui Jia
Robotics
Long-Term Human Motion Prediction Using Spatio-Temporal Maps of Dynamics
Yufei Zhu, Andrey Rudenko, Tomasz P. Kucner et al. (+2 more)
Long-term human motion prediction (LHMP) is important for the safe and efficient operation of autonomous
robots and vehicles in environments shared with humans. Accurate predictions are important for applications including motion planning, tracking, human-
robot interaction, and safety monitoring. In this paper, we exploit Maps of Dynamics (MoDs), which encode spatial or spatio-temporal motion patterns as environment features, to achieve LHMP for horizons of up to 60 seconds. We propose an MoD-informed LHMP framework that supports various types of MoDs and includes a ranking method to output the most likely predicted trajectory, improving practical utility in robotics. Further, a time-conditioned MoD is introduced to capture motion patterns that vary across different times of day. We evaluate MoD-LHMP instantiated with three types of MoDs. Experiments on two real-world
datasets show that MoD-informed method outperforms learning-based ones, with up to 50% improvement in average displacement error, and the time-conditioned variant achieves the highest accuracy overall. Project code is available at https://github.com/test-bai-cpu/LHMP-with-MoDs.git
Read the full abstract ▾
All authors Yufei Zhu, Andrey Rudenko, Tomasz P. Kucner, Achim J. Lilienthal, Martin Magnusson
Robotics
HumanoidExo: Scalable Whole-Body Humanoid Manipulation via Wearable Exoskeleton
Rui Zhong, Yizhe Sun, Junjie Wen et al. (+7 more)
A significant bottleneck in
humanoid policy learning is the acquisition of large-scale, diverse
datasets, as collecting reliable real-world data remains both difficult and cost-prohibitive. To address this limitation, we introduce HumanoidExo, a novel system that transfers human motion to whole-body
humanoid data. HumanoidExo offers a high-efficiency solution that minimizes the embodiment gap between the human demonstrator and the
robot, thereby tackling the scarcity of whole-body
humanoid data. By facilitating the collection of more voluminous and diverse
datasets, our approach significantly enhances the performance of
humanoid robots in dynamic, real-world scenarios. We evaluated our method across three challenging real-world tasks: table-top manipulation, manipulation integrated with stand-squat motions, and whole-body manipulation. Our results empirically demonstrate that HumanoidExo is a crucial addition to real-
robot data, as it enables the
humanoid policy to generalize to novel environments, learn complex whole-body control from only five real-
robot demonstrations, and even acquire new skills (i.e., walking) solely from HumanoidExo data.
Read the full abstract ▾
All authors Rui Zhong, Yizhe Sun, Junjie Wen, Jinming Li, Chuang Cheng, Wei Dai, Zhiwen Zeng, Huimin Lu, Yichen Zhu, Yi Xu
Robotics
3D-CovDiffusion: 3D-Aware Diffusion Policy for Coverage Path Planning
Chenyuan Chen, Haoran Ding, Ran Ding et al. (+4 more)
Diffusion models have shown strong potential for
robot skill learning, yet their role in coverage path planning remains underexplored. In industrial surface processing (painting, polishing, spray coating), high coverage requires globally ordered, temporally coherent trajectories rather than stitching unordered local segments. We reformulate coverage path planning as conditional sequence generation and adopt a geometry-conditioned
diffusion framework that synthesizes continuous trajectories directly from raw 3D point clouds. Our method produces temporally ordered trajectory chunks and avoids post-hoc heuristic ordering or stitching in prior learning-based methods via simple sequential concatenation, improving sequence-level consistency. A single shared policy generalizes across different geometries without category-specific architectures. Extensive
benchmarks show substantial gains over prior learning-based baselines: 98.2% lower point-wise Chamfer Distance (lower is better), 97.0% lower jerk (smoother trajectories), and +67.5 percentage points overlapping surface coverage on average.
Read the full abstract ▾
All authors Chenyuan Chen, Haoran Ding, Ran Ding, Tianyu Liu, Zewen He, Anqing Duan, Yoshihiko Nakamura
Robotics
Real-Time Nonlinear Model Predictive Control of Heavy-Duty Skid-Steered Mobile Platform for Trajectory Tracking Tasks
Alvaro Paz, Pauli Mustalahti, Mohammad Dastranj et al. (+1 more)
This paper presents a framework for real-time optimal controlling of a heavy-duty skid-steered mobile platform for trajectory tracking. The importance of accurate real-time performance of the controller lies in safety considerations of situations where the dynamic system under control is affected by uncertainties and disturbances, and the controller should compensate for such phenomena in order to provide stable performance. A multiple-shooting nonlinear model-predictive control framework is proposed in this paper. This framework benefits from suitable
algorithm along with readings from various sensors for genuine real-time performance with extremely high accuracy. The controller is then tested for tracking different trajectories where it demonstrates highly desirable performance in terms of both speed and accuracy. This controller shows remarkable improvement when compared to existing nonlinear model-predictive controllers in the literature that were implemented on skid-steered mobile platforms.
Read the full abstract ▾
All authors Alvaro Paz, Pauli Mustalahti, Mohammad Dastranj, Jouni Mattila
Robotics
AI-Enhanced Kinematic Modeling of Flexible Manipulators Using Multi-IMU Sensor Fusion
Amir Hossein Barjini and Jouni Mattila
This paper presents a novel framework for estimating the position and orientation of flexible manipulators undergoing vertical motion using multiple inertial measurement units (IMUs), optimized and calibrated with ground truth data. The flexible links are modeled as a series of rigid segments, with joint angles estimated from accelerometer and gyroscope measurements acquired by cost-effective IMUs. A complementary filter is employed to fuse the measurements, with its parameters optimized through particle swarm optimization (PSO) to mitigate noise and delay. To further improve estimation accuracy, residual errors in position and orientation are compensated using radial basis function neural networks (RBFNN). Experimental results validate the effectiveness of the proposed intelligent multi-IMU kinematic estimation method, achieving root mean square errors (RMSE) of 0.00021~m, 0.00041~m, and 0.00024~rad for $y$, $z$, and $\theta$, respectively.
Read the full abstract ▾
Robotics
YawSitter: Modeling and Controlling a Tail-Sitter UAV with Enhanced Yaw Control
Amir Habel, Fawad Mehboob, Jeffrin Sam et al. (+2 more)
Achieving precise lateral motion modeling and decoupled control in hover remains a significant challenge for tail-sitter Unmanned Aerial Vehicles (
UAVs), primarily due to complex aerodynamic couplings and the absence of welldefined lateral dynamics. This paper presents a novel modeling and control strategy that enhances yaw authority and lateral motion by introducing a sideslip force model derived from differential propeller slipstream effects acting on the fuselage under differential thrust. The resulting lateral force along the body y-axis enables yaw-based lateral position control without inducing roll coupling. The control framework employs a YXZ Euler rotation formulation to accurately represent attitude and incorporate gravitational components while directly controlling yaw in the yaxis, thereby improving lateral dynamic behavior and avoiding singularities. The proposed approach is validated through trajectory-tracking simulations conducted in a Unity-based environment. Tests on both rectangular and circular paths in hover mode demonstrate stable performance, with low mean absolute position errors and yaw deviations constrained within 5.688 degrees. These results confirm the effectiveness of the proposed lateral force generation model and provide a foundation for the development of agile, hover-capable tail-sitter
UAVs.
Read the full abstract ▾
All authors Amir Habel, Fawad Mehboob, Jeffrin Sam, Clement Fortin, Dzmitry Tsetserukou
Robotics
Single-Rod Brachiation Robot: Mechatronic Control Design and Validation of Prejump Phases
Juraj Lieskovský, Hijiri Akahane, Aoto Osawa et al. (+3 more)
A complete mechatronic design of a minimal configuration brachiation
robot is presented. The
robot consists of a single rigid rod with gripper mechanisms attached to both ends. The grippers are used to hang the
robot on a horizontal bar on which it swings or rotates. The motion is imposed by repositioning the
robot's center of mass, which is performed using a crank-slide mechanism. Based on a non-linear model, an optimal control strategy is proposed, for repositioning the center of mass in a bang-bang manner. Consequently, utilizing the concept of input-output linearization, a continuous control strategy is proposed that takes into account the limited torque of the crank-slide mechanism and its geometry. An increased
attention is paid to energy accumulation towards the subsequent jump stage of the brachiation. These two strategies are validated and compared in simulations. The continuous control strategy is then also implemented within a low-cost STM32-based control system, and both the swing and rotation stages of the brachiation motion are experimentally validated.
Read the full abstract ▾
All authors Juraj Lieskovský, Hijiri Akahane, Aoto Osawa, Jaroslav Bušek, Ikuo Mizuuchi, Tomáš Vyhlídal
Robotics
Metrics vs Surveys: An Analysis for Human-Aligned Benchmarking in Social Robot Navigation
Stefano Trepella, Mauro Martini, Noé Pérez-Higueras et al. (+4 more)
Social, also called human-aware,
navigation is a key challenge for integrating mobile
robots into human environments. The evaluation of such systems is complex, as factors such as comfort, safety, and legibility must be considered. Human-centered assessments, typically conducted through surveys, provide reliable insights but are costly, resource-intensive, and difficult to reproduce or compare across systems. Alternatively, numerical social
navigation metrics are easy to compute and facilitate comparisons, yet the community lacks consensus on a standard set of metrics. This work explores the relationship between numerical metrics and human-centered evaluations to identify potential correlations. If specific quantitative measures align with human perceptions, they could serve as preliminary benchmarking tools, providing a human-aligned assessment when large-scale surveys are not feasible. Our results indicate that while current metrics capture some aspects of
robot navigation behavior, important subjective factors remain insufficiently represented, necessitating new metrics.
Read the full abstract ▾
All authors Stefano Trepella, Mauro Martini, Noé Pérez-Higueras, Andrea Ostuni, Fernando Caballero, Luis Merino, Marcello Chiaberge
Robotics
Point Cloud-Based Control Barrier Functions for Model Predictive Control in Safety-Critical Navigation of Autonomous Mobile Robots
Faduo Liang, Yunfeng Yang and Shi-Lu Dai
In this work, we propose a novel motion planning
algorithm to facilitate safety-critical
navigation for autonomous mobile
robots. The proposed
algorithm integrates a real-time dynamic obstacle tracking and mapping system that categorizes point clouds into dynamic and static components. For dynamic point clouds, the Kalman filter is employed to estimate and predict their motion states. Based on these predictions, we extrapolate the future states of dynamic point clouds, which are subsequently merged with static point clouds to construct the forward-time-domain (FTD) map. By combining control barrier functions (CBFs) with nonlinear model predictive control, the proposed
algorithm enables the
robot to effectively avoid both static and dynamic obstacles. The CBF constraints are formulated based on risk points identified through collision detection between the predicted future states and the FTD map. Experimental results from both simulated and real-world scenarios demonstrate the efficacy of the proposed
algorithm in complex environments. In simulation experiments, the proposed
algorithm is compared with two baseline approaches, showing superior performance in terms of safety and robustness in obstacle avoidance. The source code is released for the reference of the robotics community.
Read the full abstract ▾
Robotics
Novel UWB Synthetic Aperture Radar Imaging for Mobile Robot Mapping
Charith Premachandra and U-Xuan Tan
Traditional exteroceptive sensors in mobile
robots, such as
LiDARs and cameras often struggle to perceive the environment in poor visibility conditions. Recently, radar technologies, such as ultra-wideband (UWB) have emerged as potential alternatives due to their ability to see through adverse environmental conditions (e.g.
dust, smoke and rain). However, due to the small apertures with low directivity, the UWB radars cannot reconstruct a detailed image of its field of view (FOV) using a single scan. Hence, a virtual large aperture is synthesized by moving the radar along a mobile
robot path. The resulting synthetic aperture radar (SAR) image is a high-definition representation of the surrounding environment. Hence, this paper proposes a pipeline for mobile
robots to incorporate UWB radar-based SAR imaging to map an unknown environment. Finally, we evaluated the performance of classical feature detectors: SIFT, SURF, BRISK, AKAZE and ORB to identify loop closures using UWB SAR images. The experiments were conducted emulating adverse environmental conditions. The results demonstrate the viability and effectiveness of UWB SAR imaging for high-resolution environmental mapping and loop closure detection toward more robust and reliable robotic perception systems.
Read the full abstract ▾
Robotics
Action Deviation-Aware Inference for Low-Latency Wireless Robots
Jeyoung Park, Yeonsub Lim, Seungeun Oh et al. (+3 more)
To support latency-sensitive AI applications ranging from autonomous driving to industrial
robot manipulation, 6G envisions distributed ML with computational resources in mobile, edge, and cloud connected over hyper-reliable low-latency communication (HRLLC). In this setting, speculative decoding can facilitate collaborative inference of models distributively deployed: a lightweight on-device model locally generates drafts while a more capable remote target model on a server verifies and corrects them in parallel with speculative sampling, thus resulting in lower latency without compromising accuracy. However, unlike autoregressive text generation, behavior cloning policies, typically used for
embodied AI applications, cannot parallelize verification and correction for multiple drafts as each generated action depends on observation updated by a previous action. To this end, we propose Action Deviation-Aware Hybrid Inference (ADAHI), wherein drafts are selectively transmitted and verified based on action deviation, which has a strong correlation with action's rejection probability by the target model. By invoking server operation only when necessary, communication and computational overhead can be reduced while accuracy gain from speculative sampling is preserved. Experiments on our testbed show that ADAHI reduces transmission and server operations by approximately 40%, lowers end-to-end latency by 39.2%, and attains up to 97.2% of the task-success rate of baseline that invokes speculative sampling for every draft
embedding vector.
Read the full abstract ▾
All authors Jeyoung Park, Yeonsub Lim, Seungeun Oh, Jihong Park, Jinho Choi, Seong-Lyun Kim
Robotics
Assist-as-needed Control for FES in Foot Drop Management
Andreas Christou, Elliot Lister, Georgia Andreopoulou et al. (+2 more)
Foot drop is commonly managed using Functional Electrical Stimulation (FES), typically delivered via open-loop controllers with fixed stimulation intensities. While users may manually adjust the intensity through external controls, this approach risks overstimulation, leading to muscle fatigue and discomfort, or understimulation, which compromises dorsiflexion and increases fall risk. In this study, we propose a novel closed-loop FES controller that dynamically adjusts the stimulation intensity based on real-time toe clearance, providing "assistance as needed". We evaluate this system by inducing foot drop in healthy participants and comparing the effects of the closed-loop controller with a traditional open-loop controller across various walking conditions, including different speeds and surface inclinations. Kinematic data reveal that our closed-loop controller maintains adequate toe clearance without significantly affecting the joint angles of the hips, the knees, and the ankles, and while using significantly lower stimulation intensities compared to the open-loop controller. These findings suggest that the proposed method not only matches the effectiveness of existing systems but also offers the potential for reduced muscle fatigue and improved long-term user comfort and adherence.
Read the full abstract ▾
All authors Andreas Christou, Elliot Lister, Georgia Andreopoulou, Don Mahad, Sethu Vijayakumar
Robotics
Work Zones challenge VLM Trajectory Planning: Toward Mitigation and Robust Autonomous Driving
Yifan Liao, Zhen Sun, Xiaoyun Qiu et al. (+7 more)
Visual
Language Models (VLMs), with powerful
multimodal reasoning capabilities, are gradually integrated into autonomous driving by several automobile manufacturers to enhance planning capability in challenging environments. However, the trajectory planning capability of VLMs in work zones, which often include irregular layouts, temporary traffic control, and dynamically changing
geometric structures, is still unexplored. To bridge this gap, we conduct the first systematic study of VLMs for work zone trajectory planning, revealing that mainstream VLMs fail to generate correct trajectories in $68.0%$ of cases. To better understand these failures, we first identify candidate patterns via subgraph mining and clustering analysis, and then confirm the validity of $8$ common failure patterns through human verification. Building on these findings, we propose REACT-Drive, a trajectory planning framework that integrates VLMs with
Retrieval-Augmented Generation (RAG). Specifically, REACT-Drive leverages VLMs to convert prior failure cases into constraint rules and executable trajectory planning code, while RAG retrieves similar patterns in new scenarios to guide trajectory generation. Experimental results on the ROADWork
dataset show that REACT-Drive yields a reduction of around $3\times$ in average displacement error relative to VLM baselines under evaluation with Qwen2.5-VL. In addition, REACT-Drive yields the lowest inference time ($0.58$s) compared with other methods such as fine-tuning ($17.90$s). We further conduct experiments using a real vehicle in 15 work zone scenarios in the physical world, demonstrating the strong practicality of REACT-Drive.
Read the full abstract ▾
All authors Yifan Liao, Zhen Sun, Xiaoyun Qiu, Zixiao Zhao, Wenbing Tang, Xinlei He, Xinhu Zheng, Tianwei Zhang, Xinyi Huang, Xingshuo Han
Robotics · Cross-listed from cs.CV
VERNIER: an open-source software pushing marker pose estimation down to the micrometer and nanometer scales
Patrick Sandoz, Antoine N. André and Guillaume J. Laurent
Pose estimation is still a challenge at the small scales. Few solutions exist to capture the 6 degrees of freedom of an object with nanometric and microradians resolutions over relatively large ranges. Over the years, we have proposed several fiducial marker and pattern designs to achieve reliable performance for various microscopy applications. Centimeter ranges are possible using pattern encoding methods, while nanometer resolutions can be achieved using phase processing of the periodic frames. This paper presents VERNIER, an open source phase processing software designed to provide fast and reliable pose measurement based on pseudo-periodic patterns. Thanks to a phase-based local thresholding
algorithm, the software has proven to be particularly robust to noise, defocus and occlusion. The successive steps of the phase processing are presented, as well as the different types of patterns that address different application needs. The implementation procedure is illustrated with synthetic and experimental images. Finally, guidelines are given for selecting the appropriate pattern design and microscope magnification lenses as a function of the desired performance.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
Periodic Event-Triggered Prescribed Time Control of Euler-Lagrange Systems under State and Input Constraints
Chidre Shravista Kashyap, Karnan A, Pushpak Jagtap et al. (+1 more)
This article proposes a periodic event-triggered adaptive barrier control policy for the trajectory tracking problem of perturbed Euler-Lagrangian systems with state, input, and temporal (SIT) constraints. In particular, an approximation-free adaptive-barrier control architecture is designed to ensure prescribed-time convergence of the tracking error to a prescribed bound while rejecting exogenous disturbances. In contrast to existing approaches that necessitate continuous real-time control action, the proposed controller generates event-based updates through periodic evaluation of the triggering condition. Additionally, we derive an upper bound on the monitoring period by analysing the performance degradation of the filtered tracking error to facilitate periodic evaluation of the event-triggered strategy. To this end, a time-varying threshold function is considered in the triggering mechanism to reduce the number of triggers during the transient phase of system behaviour. Notably, the proposed design avoids Zeno behaviour and precludes the need for continuous monitoring of the triggering condition. A simulation and experimental study is undertaken to demonstrate the efficacy of the proposed control scheme.
Read the full abstract ▾
All authors Chidre Shravista Kashyap, Karnan A, Pushpak Jagtap, Jishnu Keshavan
Robotics
Flow with the Force Field: Learning 3D Compliant Flow Matching Policies from Force and Demonstration-Guided Simulation Data
Tianyu Li, Yihan Li, Zizhe Zhang et al. (+1 more)
While visuomotor policy has made advancements in recent years, contact-rich tasks still remain a challenge. Robotic manipulation tasks that require continuous contact demand explicit handling of compliance and force. However, most visuomotor policies ignore compliance, overlooking the importance of physical interaction with the real world, often leading to excessive contact forces or fragile behavior under uncertainty. Introducing force information into
vision-based imitation learning could help improve awareness of contacts, but could also require a lot of data to perform well. One remedy for data scarcity is to generate data in simulation, yet computationally taxing processes are required to generate data good enough not to suffer from the Sim2Real gap. In this work, we introduce a framework for generating force-informed data in simulation, instantiated by a single human demonstration, and show how coupling with a compliant policy improves the performance of a visuomotor policy learned from synthetic data. We validate our approach on real-
robot tasks, including non-prehensile block flipping and a bi-manual object moving, where the learned policy exhibits reliable contact maintenance and adaptation to novel conditions. Project Website: https://flow-with-the-force-field.github.io/webpage/
Read the full abstract ▾
All authors Tianyu Li, Yihan Li, Zizhe Zhang, Nadia Figueroa
Robotics
Team Xiaomi EV-AD VLA: Caption-Guided Retrieval System for Cross-Modal Drone Navigation – Technical Report for IROS 2025 RoboSense Challenge Track 4
Lingfeng Zhang, Erjia Xiao, Yuchen Zhang et al. (+7 more)
Cross-modal drone
navigation remains a challenging task in robotics, requiring efficient
retrieval of relevant images from large-scale databases based on natural
language descriptions. The RoboSense 2025 Track 4 challenge addresses this challenge, focusing on robust, natural
language-guided cross-view image
retrieval across multiple platforms (drones, satellites, and ground cameras). Current baseline methods, while effective for initial
retrieval, often struggle to achieve fine-grained semantic matching between text queries and visual content, especially in complex aerial scenes. To address this challenge, we propose a two-stage
retrieval refinement method: Caption-Guided
Retrieval System (CGRS) that enhances the baseline coarse ranking through intelligent reranking. Our method first leverages a baseline model to obtain an initial coarse ranking of the top 20 most relevant images for each query. We then use
Vision-
Language-Model (VLM) to generate detailed captions for these candidate images, capturing rich semantic descriptions of their visual content. These generated captions are then used in a
multimodal similarity computation framework to perform fine-grained reranking of the original text query, effectively building a semantic bridge between the visual content and natural
language descriptions. Our approach significantly improves upon the baseline, achieving a consistent 5% improvement across all key metrics (Recall@1, Recall@5, and Recall@10). Our approach win TOP-2 in the challenge, demonstrating the practical value of our semantic refinement strategy in real-world robotic
navigation scenarios.
Read the full abstract ▾
All authors Lingfeng Zhang, Erjia Xiao, Yuchen Zhang, Haoxiang Fu, Ruibin Hu, Yanbiao Ma, Wenbo Ding, Long Chen, Hangjun Ye, Xiaoshuai Hao