Robotics
Real-time Mixed-Integer Quadratic Programming for Driving Behavior-Inspired Speed Bump Optimal Trajectory Planning
Van Nam Dinh, Van Vy Phan, Thai Son Dang et al. (+6 more)
This paper proposes a novel methodology for trajectory planning in autonomous vehicles (AVs), addressing the complex challenge of negotiating speed bumps within a unified Mixed-Integer Quadratic Programming (MIQP) framework. By leveraging Model Predictive Control (MPC), we develop trajectories that optimize both the traversal of speed bumps and overall passenger comfort. A key contribution of this work is the formulation of speed bump handling constraints that closely emulate human driving behavior, seamlessly integrating these with broader road
navigation requirements. Through extensive simulations in varied urban driving environments, we demonstrate the efficacy of our approach, highlighting its ability to ensure smooth speed transitions over speed bumps while maintaining computational efficiency suitable for real-time deployment. The method's capability to handle both static road features and dynamic constraints, alongside expert human driving, represents a significant step forward in trajectory planning for urban
Read the full abstract ▾
All authors Van Nam Dinh, Van Vy Phan, Thai Son Dang, Van Du Phan, The Anh Mai, Van Chuong Le, Sy Phuong Ho, Dinh Tu Duong, Hung Cuong Ta
Robotics · Cross-listed from eess.SY
Computing Safe Control Inputs using Discrete-Time Matrix Control Barrier Functions via Convex Optimization
James Usevitch, Juan Augusto Paredes Salazar and Ankit Goel
Control barrier functions (CBFs) have seen widespread success in providing forward invariance and safety guarantees for dynamical control systems. A crucial limitation of discrete-time formulations is that CBFs that are nonconcave in their argument require the solution of nonconvex optimization problems to compute safety-preserving control inputs, which inhibits real-time computation of control inputs guaranteeing forward invariance. This paper presents a novel method for computing safety-preserving control inputs for discrete-time systems with nonconvex safety sets, utilizing convex optimization and the recently developed class of
matrix control barrier function techniques. The efficacy of our methods is demonstrated through numerical simulations on a bicopter system.
Read the full abstract ▾
Robotics
Cross-Sensor Touch Generation
Samanta Rodriguez, Yiming Dou, Miquel Oller et al. (+2 more)
Today's visuo-tactile sensors come in many shapes and sizes, making it challenging to develop general-purpose tactile representations. This is because most models are tied to a specific sensor design. To address this challenge, we propose two approaches to cross-sensor image generation. The first is an end-to-end method that leverages paired data (Touch2Touch). The second method builds an intermediate depth representation and does not require paired data (T2D2: Touch-to-Depth-to-Touch). Both methods enable the use of sensor-specific models across multiple sensors via the cross-sensor touch generation process. Together, these models offer flexible solutions for sensor translation, depending on data availability and application needs. We demonstrate their effectiveness on downstream tasks such as in-hand pose estimation and behavior cloning, successfully transferring models trained on one sensor to another. Project page: https://samantabelen.github.io/cross_sensor_touch_generation.
Read the full abstract ▾
All authors Samanta Rodriguez, Yiming Dou, Miquel Oller, Andrew Owens, Nima Fazeli
Robotics
Enhancing Diffusion Policy with Classifier-Free Guidance for Temporal Robotic Tasks
Yuang Lu, Song Wang, Xiao Han et al. (+3 more)
Temporal sequential tasks challenge
humanoid robots, as existing
Diffusion Policy (DP) and Action Chunking with
Transformers (ACT) methods often lack temporal context, resulting in local optima traps and excessive repetitive actions. To address these issues, this paper introduces a Classifier-Free Guidance-Based
Diffusion Policy (CFG-DP), a novel framework to enhance DP by integrating Classifier-Free Guidance (CFG) with conditional and unconditional models. Specifically, CFG leverages timestep inputs to track task progression and ensure precise cycle termination. It dynamically adjusts action predictions based on task phase, using a guidance factor tuned to balance temporal coherence and action accuracy. Real-world experiments on a
humanoid robot demonstrate high success rates and minimal repetitive actions. Furthermore, we assessed the model's ability to terminate actions and examined how different components and parameter adjustments affect its performance. This framework significantly enhances deterministic control and execution reliability for sequential robotic tasks.
Read the full abstract ▾
All authors Yuang Lu, Song Wang, Xiao Han, Xuri Zhang, Yucong Wu, Zhicheng He
Robotics
Online Structure Learning and Planning for Autonomous Robot Navigation using Active Inference
Daria de tinguy, Tim Verbelen, Emilio Gamba et al. (+1 more)
Autonomous
navigation in unfamiliar environments requires
robots to simultaneously explore, localise, and plan under uncertainty, without relying on predefined maps or extensive training. We present Active Inference MAPping and Planning (AIMAPP), a framework unifying mapping, localisation, and decision-making within a single generative model, drawing on cognitive-mapping concepts from animal
navigation (topological organisation, discrete spatial representations and predictive belief updating) as design inspiration. The
agent builds and updates a sparse topological map online, learns state transitions dynamically, and plans actions by minimising Expected Free Energy. This allows it to balance goal-directed and exploratory behaviours. We implemented AIMAPP as a ROS-compatible system that is sensor and
robot-agnostic and integrates with diverse hardware configurations. It operates in a fully self-supervised manner, is resilient to sensor failure, continues operating under odometric drift, and supports both exploration and goal-directed
navigation without any pre-training. We evaluate the system in large-scale real and simulated environments against state-of-the-art planning baselines, demonstrating its adaptability to ambiguous observations, environmental changes, and sensor noise. The model offers a modular, self-supervised solution to scalable
navigation in unstructured settings. AIMAPP is available at https://github.com/decide-ugent/aimapp.
Read the full abstract ▾
All authors Daria de tinguy, Tim Verbelen, Emilio Gamba, Bart Dhoedt
Robotics · Cross-listed from cs.HC
Differential Analysis of Pseudo Haptic Feedback: Novel Comparative Study of Visual and Auditory Cue Integration for Psychophysical Evaluation
Nishant Gautam, Somya Sharma, Peter Corcoran et al. (+1 more)
Pseudo-haptics exploit carefully crafted visual or auditory cues to trick the
brain into "feeling" forces that are never physically applied, offering a low-cost alternative to traditional haptic hardware. Here, we present a comparative psychophysical study that quantifies how visual and auditory stimuli combine to evoke pseudo-haptic pressure sensations on a commodity tablet. Using a Unity-based Rollball game, participants (n = 4) guided a virtual ball across three textured terrains while their finger forces were captured in real time with a Robotous RFT40 force-torque sensor. Each terrain was paired with a distinct rolling-sound profile spanning 440 Hz - 4.7 kHz, 440 Hz - 13.1 kHz, or 440 Hz - 8.9 kHz; crevice collisions triggered additional "knocking" bursts to heighten realism. Average tactile forces increased systematically with cue intensity: 0.40 N, 0.79 N and 0.88 N for visual-only trials and 0.41 N, 0.81 N and 0.90 N for audio-only trials on Terrains 1-3, respectively. Higher audio frequencies and denser visual textures both elicited stronger muscle activation, and their combination further reduced the force needed to perceive surface changes, confirming multisensory integration. These results demonstrate that consumer-grade isometric devices can reliably induce and measure graded pseudo-haptic feedback without specialized actuators, opening a path toward affordable rehabilitation tools, training simulators and assistive interfaces.
Read the full abstract ▾
All authors Nishant Gautam, Somya Sharma, Peter Corcoran, Kaspar Althoefer
Robotics
Guiding Energy-Efficient Locomotion through Impact Mitigation Rewards
Chenghao Wang, Arjun Viswanathan, Eric Sihite et al. (+1 more)
Animals achieve energy-efficient locomotion by their implicit passive dynamics, a marvel that has captivated roboticists for decades.Recently, methods incorporated
Adversarial Motion Prior (AMP) and Reinforcement learning (RL) shows promising progress to replicate Animals' naturalistic motion. However, such imitation learning approaches predominantly capture explicit kinematic patterns, so-called gaits, while overlooking the implicit passive dynamics. This work bridges this gap by incorporating a reward term guided by Impact Mitigation Factor (IMF), a physics-informed metric that quantifies a
robot's ability to passively mitigate impacts. By integrating IMF with AMP, our approach enables RL policies to learn both explicit motion trajectories from animal reference motion and the implicit passive dynamic. We demonstrate energy efficiency improvements of up to 32%, as measured by the Cost of Transport (CoT), across both AMP and handcrafted reward structure.
Read the full abstract ▾
All authors Chenghao Wang, Arjun Viswanathan, Eric Sihite, Alireza Ramezani
Robotics
Dynamic Quadrupedal Legged and Aerial Locomotion via Structure Repurposing
Chenghao Wang, Kaushik Venkatesh Krishnamurthy, Shreyansh Pitroda et al. (+5 more)
Multi-modal ground-aerial
robots have been extensively studied, with a significant challenge lying in the integration of conflicting requirements across different modes of operation. The Husky
robot family, developed at Northeastern University, and specifically the Husky v.2 discussed in this study, addresses this challenge by incorporating posture manipulation and thrust vectoring into multi-modal locomotion through structure repurposing. This quadrupedal
robot features leg structures that can be repurposed for dynamic legged locomotion and flight. In this paper, we present the hardware design of the
robot and report primary results on dynamic quadrupedal legged locomotion and hovering.
Read the full abstract ▾
All authors Chenghao Wang, Kaushik Venkatesh Krishnamurthy, Shreyansh Pitroda, Adarsh Salagame, Ioannis Mandralis, Eric Sihite, Alireza Ramezani, Morteza Gharib
Robotics · Cross-listed from cond-mat.soft
Toggling stiffness via multistability
Hugo de Souza Oliveira, Michele Curatolo, Renate Sachse et al. (+1 more)
Variable stiffness is a key capability in biological and robotic systems, enabling adaptive interaction across tasks and environments. Mechanical metamaterials offer an alternative to conventional mechatronic solutions by encoding stiffness variation directly into monolithic structural architectures, reducing the need for discrete assemblies. Here, we introduce a multistable mechanical metamaterial that exhibits a toggleable stiffness effect in which the effective shear stiffness switches discretely between stable mechanical configurations. Mechanical analysis of surrogate beam models of the unit
cell reveals that this behavior originates from the rotation transmitted by the support beams to the curved beam, governing the balance between bending and axial deformation. Consequently, the shear stiffness ratio between the two states can be tuned by varying the slenderness of the support beams or by incorporating localized hinges that modulate rotational transfer. Experiments on 3D-printed prototypes validate the numerical predictions and confirm consistent stiffness toggling across different geometries. Finally, we demonstrate a monolithic soft clutch that leverages this effect to achieve programmable, stepwise stiffness modulation. This work establishes a design strategy for toggleable stiffness using multistable metamaterials, with potential applications in soft robotics and smart structures where adaptive compliance is of paramount importance.
Read the full abstract ▾
All authors Hugo de Souza Oliveira, Michele Curatolo, Renate Sachse, Edoardo Milana
Robotics · Cross-listed from cs.CV
PhysToolBench: Benchmarking Physical Tool Understanding for MLLMs
Zixin Zhang, Kanghao Chen, Xingwang Lin et al. (+6 more)
The ability to use, understand, and create tools is a hallmark of human intelligence, enabling sophisticated interaction with the physical world. For any general-purpose intelligent
agent to achieve true versatility, it must also master these fundamental skills. While modern
Multimodal Large
Language Models (MLLMs) leverage their extensive common knowledge for high-level planning in
embodied AI and in downstream
Vision-
Language-Action (VLA) models, the extent of their true understanding of physical tools remains unquantified. To bridge this gap, we present PhysToolBench, the first
benchmark dedicated to evaluating the comprehension of physical tools by MLLMs. Our
benchmark is structured as a Visual Question Answering (VQA)
dataset comprising over 1,000 image-text pairs. It assesses capabilities across three distinct difficulty levels: (1) Tool Recognition: Requiring the recognition of a tool's primary function. (2) Tool Understanding: Testing the ability to grasp the underlying principles of a tool's operation. (3) Tool Creation: Challenging the model to fashion a new tool from surrounding objects when conventional options are unavailable. Our comprehensive evaluation of 32 MLLMs-spanning proprietary, open-source, specialized
embodied, and backbones in VLAs-reveals a significant deficiency in tool understanding. Furthermore, we provide an in-depth analysis and propose preliminary solutions. Code and
dataset are publicly available.
Read the full abstract ▾
All authors Zixin Zhang, Kanghao Chen, Xingwang Lin, Lutao Jiang, Xu Zheng, Yuanhuiyi Lyu, Litao Guo, Yinchuan Li, Ying-Cong Chen
Robotics
Toward Autonomous Soft Robotic Endovascular Navigation via Imitation Learning
Noah Barnes, Ji Woong Kim, Lingyun Di et al. (+11 more)
In endovascular surgery, endovascular interventionists push a thin tube called a catheter, guided by a thin wire to a treatment site inside the patient's blood vessels to treat various conditions such as blood clots, aneurysms, and malformations. Robotic guidewires can enhance maneuverability but are difficult to model and control. Autonomous soft robotic guidewire
navigation has the potential to overcome these challenges, increasing the precision and safety of endovascular
navigation. As a first step, we establish a large-scale, 2D-projected environment for autonomous
navigation. In other
surgical domains, end-to-end imitation learning has shown promising results. Thus, we develop a
transformer-based imitation learning framework with goal conditioning, relative action outputs, and automatic contrast dye injections to enable generalizable soft
robot navigation in an aneurysm targeting task. We train the policy on 36 different modular bifurcated geometries, generating 647 total demonstrations under simulated fluoroscopy, and evaluate it on three previously unseen vascular geometries. The policy reaches the aneurysm with a success rate of 83% on the unseen geometries, outperforming several baselines. In addition, ablation and baseline studies evaluate the effectiveness of each design and data collection choice. Lastly, we extend the policy to achieve 75% success on an unseen patient-derived geometry. Project website: https://softrobotnavigation.github.io/
Read the full abstract ▾
All authors Noah Barnes, Ji Woong Kim, Lingyun Di, Hannah Qu, Anuruddha Bhattacharjee, Miroslaw Janowski, Dheeraj Gandhi, Bailey Felix, Shaopeng Jiang, Olivia Young, Mark Fuge, Ryan D. Sochol, Jeremy D. Brown, Axel Krieger
Robotics
FOGMACHINE – Leveraging Discrete-Event Simulation and Scene Graphs for Modeling Hierarchical, Interconnected Environments under Partial Observations from Mobile Agents
Lars Ohnemus, Nils Hantke, Max Weißer et al. (+1 more)
Dynamic Scene Graphs (DSGs) provide a structured representation of hierarchical, interconnected environments, but current approaches struggle to capture
stochastic dynamics, partial observability, and multi-
agent activity. These aspects are critical for
embodied AI, where
agents must act under uncertainty and delayed perception. We introduce FOGMACHINE , an open-source framework that fuses DSGs with discrete-event simulation to model object dynamics,
agent observations, and interactions at scale. This setup enables the study of uncertainty propagation, planning under limited perception, and emergent multi-
agent behavior. Experiments in urban scenarios illustrate realistic temporal and spatial patterns while revealing the challenges of belief estimation under sparse observations. By combining structured representations with efficient simulation, FOGMACHINE establishes an effective tool for benchmarking, model training, and advancing
embodied AI in complex, uncertain environments.
Read the full abstract ▾
All authors Lars Ohnemus, Nils Hantke, Max Weißer, Kai Furmans
Robotics · Cross-listed from cs.MA
Towards Information-Optimized Multi-Agent Path Finding: A Hybrid Framework with Reduced Inter-Agent Information Sharing
Bharath Muppasani, Ritirupa Dey, Biplav Srivastava et al. (+1 more)
Multi-
agent pathfinding (MAPF) remains a critical problem in robotics and autonomous systems, where
agents must navigate shared spaces efficiently while avoiding conflicts. Traditional centralized
algorithms with global information provide high-quality solutions but scale poorly in large-scale scenarios due to the combinatorial explosion of conflicts. Conversely, distributed approaches that have local information, particularly learning-based methods, offer better scalability by operating with relaxed information availability, yet often at the cost of solution quality. In realistic deployments, information is a constrained resource: broadcasting full
agent states and goals can raise
privacy concerns, strain limited bandwidth, and require extra sensing and communication hardware, increasing cost and energy use. We focus on the core question of how MAPF can be solved with minimal inter-
agent information sharing while preserving solution feasibility. To this end, we present an information-centric formulation of the MAPF problem and introduce a hybrid framework, IO-MAPF, that integrates decentralized path planning with a lightweight centralized coordinator. In this framework,
agents use reinforcement learning (RL) to plan independently, while the central coordinator provides minimal, targeted signals, such as static conflict-
cell indicators or short conflict trajectories, that are dynamically shared to support efficient conflict resolution. We introduce an Information Units (IU) metric to quantify information use and show that our alert-driven design achieves 2x to 23x reduction in information sharing, compared to the state-of-the-art
algorithms, while maintaining high success rates, demonstrating that reliable MAPF is achievable under strongly information-restricted,
privacy-preserving conditions. We demonstrate the effectiveness of our
algorithm using simulation and hardware experiments.
Read the full abstract ▾
All authors Bharath Muppasani, Ritirupa Dey, Biplav Srivastava, Vignesh Narayanan
Robotics
Failure Prediction at Runtime for Generative Robot Policies
Ralf Römer, Adrian Kobras, Luca Worbis et al. (+1 more)
Imitation learning (IL) with generative models, such as
diffusion and flow matching, has enabled
robots to perform complex, long-horizon tasks. However, distribution shifts from unseen environments or compounding action errors can still cause unpredictable and unsafe behavior, leading to task failure. Early failure prediction during runtime is therefore essential for deploying
robots in human-centered and safety-critical environments. We propose FIPER, a general framework for Failure Prediction at Runtime for generative IL policies that does not require failure data. FIPER identifies two key indicators of impending failure: (i) out-of-distribution (OOD) observations detected via random network
distillation in the policy's
embedding space, and (ii) high uncertainty in generated actions measured by a novel action-chunk
entropy score. Both failure prediction scores are calibrated using a small set of successful rollouts via conformal prediction. A failure alarm is triggered when both indicators, aggregated over short time windows, exceed their thresholds. We evaluate FIPER across five simulation and real-world environments involving diverse failure modes. Our results demonstrate that FIPER better distinguishes actual failures from benign OOD situations and predicts failures more accurately and earlier than existing methods. We thus consider this work an important step towards more interpretable and safer generative
robot policies. Code, data and
videos are available at https://tum-lsy.github.io/fiper_website.
Read the full abstract ▾
All authors Ralf Römer, Adrian Kobras, Luca Worbis, Angela P. Schoellig
Robotics · Cross-listed from cs.CV
SilvaScenes: Tree Detection and Species Classification from Under-Canopy Images in Natural Forests
David-Alexandre Duclos, William Guimont-Martin, Gabriel Jeanson et al. (+7 more)
Interest in forestry automation is growing alongside rapid advances in deep learning. In particular, tree detection and taxonomic classification are seen as core tasks required for automating field surveys and forestry equipment. These operations must often be performed in under-canopy settings, which pose challenging conditions for perception systems, including heavy occlusion, variable lighting, and dense vegetation. Despite this necessity, current work has yet to properly establish the feasibility of simultaneously executing tree detection and taxonomic classification in natural forests, as available
datasets primarily focus on urban settings or on a limited number of species. To address this gap, we present SilvaScenes, a
benchmark dataset for instance
segmentation of tree species from under-canopy images in natural forests. Collected across five bioclimatic domains in Quebec, Canada, our
dataset features 1421 trees from 28 species, with
segmentation masks for pixel-precise tree trunk detection and fine-grained species annotations from forestry experts. We demonstrate the relevance and difficult nature of SilvaScenes by evaluating modern deep learning approaches, showing that while trunk
segmentation is feasible, with a top mean average precision (mAP) of 69.9% and mean average recall (mAR) of 76.4%, species-aware
segmentation remains a significant challenge with an mAP and an mAR of only 39.2% and 68.6%, respectively. Alongside additional experiments, we highlight key challenges, namely that species imbalance and tree occlusion figure among the most pressing issues for precise
segmentation and identification. Meanwhile, higher image resolutions contribute to significant performance gains and will likely prove fundamental to these tasks moving forward. Our
dataset, source code, and models will be made available at https://github.com/norlab-ulaval/SilvaScenes.
Read the full abstract ▾
All authors David-Alexandre Duclos, William Guimont-Martin, Gabriel Jeanson, Arthur Larochelle-Tremblay, Martine Lapointe, Théo Defosse, Frédéric Moore, Philippe Nolet, François Pomerleau, Philippe Giguère
Robotics
Bridging Research and Practice in Simulation-based Testing of Industrial Robot Navigation Systems
Sajad Khatiri, Francisco Eli Vina Barrientos, Maximilian Wulf et al. (+2 more)
Ensuring robust robotic
navigation in dynamic environments is a key challenge, as traditional testing methods often struggle to cover the full spectrum of operational requirements. This paper presents the industrial adoption of Surrealist, a simulation-based test generation framework originally for
UAVs, now applied to the ANYmal quadrupedal
robot for industrial inspection. Our method uses a search-based
algorithm to automatically generate challenging obstacle avoidance scenarios, uncovering failures often missed by manual testing. In a pilot phase, generated test suites revealed critical weaknesses in one experimental
algorithm (40.3% success rate) and served as an effective
benchmark to prove the superior robustness of another (71.2% success rate). The framework was then integrated into the ANYbotics workflow for a six-month industrial evaluation, where it was used to test five proprietary
algorithms. A formal survey confirmed its value, showing it enhances the development process, uncovers critical failures, provides objective
benchmarks, and strengthens the overall verification pipeline.
Read the full abstract ▾
All authors Sajad Khatiri, Francisco Eli Vina Barrientos, Maximilian Wulf, Paolo Tonella, Sebastiano Panichella
Robotics · Cross-listed from math.AT
Parametrized Topological Complexity for a Multi-Robot System with Variable Tasks
Gopal Chandra Dutta, Amit Kumar Paul and Subhankar Sau
We study a generalized motion planning problem involving multiple autonomous
robots navigating in a $d$-dimensional Euclidean space in the presence of a set of obstacles whose positions are unknown a priori. Each
robot is required to visit sequentially a prescribed set of target states, with the number of targets varying between
robots. This heterogeneous setting generalizes the framework considered in the prior works on sequential parametrized topological complexity by Farber and the second author of this article. To determine the topological complexity of our problem, we formulate it mathematically by constructing an appropriate fibration. Our main contribution is the determination of this invariant in the generalized setting, which captures the minimal algorithmic instability required for designing collision-free motion planning
algorithms under parameter-dependent constraints. We provide a detailed analysis for both odd and even-dimensional ambient spaces, including the essential cohomological computations and explicit constructions of corresponding motion planning
algorithms.
Read the full abstract ▾
Robotics
Placeit! A Framework for Learning Robot Object Placement Skills
Amina Ferrad, Johann Huber, François Hélénon et al. (+3 more)
Robotics research has made significant strides in learning, yet mastering basic skills like object placement remains a fundamental challenge. A key bottleneck is the acquisition of large-scale, high-quality data, which is often a manual and laborious process. Inspired by Graspit!, a foundational work that used simulation to automatically generate dexterous grasp poses, we introduce Placeit!, an evolutionary-computation framework for generating valid placement positions for rigid objects. Placeit! is highly versatile, supporting tasks from placing objects on tables to stacking and inserting them. Our experiments show that by leveraging quality-diversity optimization, Placeit! significantly outperforms state-of-the-art methods across all scenarios for generating diverse valid poses. A pick&place pipeline built on our framework achieved a 90% success rate over 120 real-world deployments. This work positions Placeit! as a powerful tool for open-environment pick-and-place tasks and as a valuable engine for generating the data needed to train simulation-based foundation models in robotics.
Read the full abstract ▾
All authors Amina Ferrad, Johann Huber, François Hélénon, Julien Gleyze, Mahdi Khoramshahi, Stéphane Doncieux
Robotics
Obstacle Avoidance using Dynamic Movement Primitives and Reinforcement Learning
Dominik Urbaniak, Alejandro Agostini, Pol Ramon et al. (+3 more)
Learning-based motion planning can quickly generate near-optimal trajectories. However, it often requires either large training
datasets or costly collection of human demonstrations. This work proposes an alternative approach that quickly generates smooth, near-optimal collision-free 3D Cartesian trajectories from a single artificial demonstration. The demonstration is encoded as a Dynamic Movement Primitive (DMP) and iteratively reshaped using policy-based reinforcement learning to create a diverse trajectory
dataset for varying obstacle configurations. This
dataset is used to train a neural network that takes as inputs the task parameters describing the obstacle dimensions and location, derived automatically from a point cloud, and outputs the DMP parameters that generate the trajectory. The approach is validated in simulation and real-
robot experiments, outperforming a RRT-Connect baseline in terms of computation and execution time, as well as trajectory length, while supporting multi-modal trajectory generation for different obstacle geometries and end-effector dimensions.
Videos and the implementation code are available at https://github.com/DominikUrbaniak/obst-avoid-dmp-pi2.
Read the full abstract ▾
All authors Dominik Urbaniak, Alejandro Agostini, Pol Ramon, Jan Rosell, Raúl Suárez, Michael Suppa
Robotics
Glovity: Learning Dexterous Contact-Rich Manipulation via Spatial Wrench Feedback Teleoperation System
Yuyang Gao, Haofei Ma and Pai Zheng
We present Glovity, a novel, low-cost wearable teleoperation system that integrates a spatial wrench (force-torque) feedback device with a haptic glove featuring fingertip
Hall sensor calibration, enabling feedback-rich dexterous manipulation. Glovity addresses key challenges in contact-rich tasks by providing intuitive wrench and tactile feedback, while overcoming embodiment gaps through precise retargeting. User studies demonstrate significant improvements: wrench feedback boosts success rates in book-flipping tasks from 48% to 78% and reduces completion time by 25%, while fingertip calibration enhances thin-object grasping success significantly compared to commercial glove. Furthermore, incorporating wrench signals into imitation learning (via DP-R3M) achieves high success rate in novel contact-rich scenarios, such as adaptive page flipping and force-aware handovers. All hardware designs, software will be open-sourced. Project website: https://glovity.github.io/
Read the full abstract ▾
Robotics
HANDO: Hierarchical Autonomous Navigation and Dexterous Omni-loco-manipulation
Jingyuan Sun, Chaoran Wang, Mingyu Zhang et al. (+6 more)
Seamless loco-manipulation in unstructured environments requires
robots to leverage autonomous exploration alongside whole-body control for physical interaction. In this work, we introduce HANDO (Hierarchical Autonomous
Navigation and Dexterous Omni-loco-manipulation), a two-layer framework designed for legged
robots equipped with manipulators to perform human-centered mobile manipulation tasks. The first layer utilizes a goal-conditioned autonomous exploration policy to guide the
robot to semantically specified targets, such as a black office chair in a dynamic environment. The second layer employs a unified whole-body loco-manipulation policy to coordinate the arm and legs for precise interaction tasks-for example, handing a drink to a person seated on the chair. We have conducted an initial deployment of the
navigation module, and will continue to pursue finer-grained deployment of whole-body loco-manipulation.
Read the full abstract ▾
All authors Jingyuan Sun, Chaoran Wang, Mingyu Zhang, Cui Miao, Hongyu Ji, Zihan Qu, Han Sun, Bing Wang, Qingyi Si
Robotics
PLEXUS Hand: Lightweight Four-Motor Prosthetic Hand Enabling Precision-Lateral Dexterous Manipulation
Yuki Kuroda, Tomoya Takahashi, Cristian C Beltran-Hernandez et al. (+2 more)
Electric prosthetic hands should be lightweight to decrease the burden on the user, shaped like human hands for cosmetic purposes, and have motors inside to protect them from damage and dirt. In addition to the ability to perform daily activities, these features are essential for everyday use of the hand. In-hand manipulation is necessary to perform daily activities such as transitioning between different postures, particularly through rotational movements, such as reorienting cards before slot insertion and operating tools such as screwdrivers. However, currently used electric prosthetic hands only achieve static grasp postures, and existing manipulation approaches require either many motors, which makes the prosthesis heavy for daily use in the hand, or complex mechanisms that demand a large internal space and force external motor placement, complicating attachment and exposing the components to damage. Alternatively, we combine a single-axis thumb and optimized thumb positioning to achieve basic posture and in-hand manipulation, that is, the reorientation between precision and lateral grasps, using only four motors in a lightweight (311 g) prosthetic hand. Experimental validation using primitive objects of various widths (5-30 mm) and shapes (cylinders and prisms) resulted in success rates of 90-100% for reorientation tasks. The hand performed seal stamping and USB device insertion, as well as rotation to operate a screwdriver.
Read the full abstract ▾
All authors Yuki Kuroda, Tomoya Takahashi, Cristian C Beltran-Hernandez, Masashi Hamaya, Kazutoshi Tanaka
Robotics
Flow-Opt: Scalable Centralized Multi-Robot Trajectory Optimization with Flow Matching and Differentiable Optimization
Simon Idoko, Prajyot Jadhav and Arun Kumar Singh
Centralized trajectory optimization in the joint space of multiple
robots allows access to a larger feasible space that can result in smoother trajectories, especially while planning in tight spaces. Unfortunately, it is often computationally intractable beyond a very small swarm size. In this paper, we propose Flow-Opt, a learning-based approach towards improving the computational tractability of centralized multi-
robot trajectory optimization. Specifically, we reduce the problem to first learning a generative model to sample different candidate trajectories and then using a learned Safety-Filter(SF) to ensure fast inference-time constraint satisfaction. We propose a flow-matching model with a
diffusion transformer (DiT) augmented with permutation invariant
robot position and map encoders as the generative model. We develop a custom solver for our SF and equip it with a neural network that predicts context-specific initialization. The initialization network is trained in a self-supervised manner, taking advantage of the differentiability of the SF solver. We advance the state-of-the-art in the following respects. First, we show that we can generate trajectories of tens of
robots in cluttered environments in a few tens of milliseconds. This is several times faster than existing centralized optimization approaches. Moreover, our approach also generates smoother trajectories orders of magnitude faster than competing baselines based on
diffusion models. Second, each component of our approach can be batched, allowing us to solve a few tens of problem instances in a fraction of a second. We believe this is a first such result; no existing approach provides such capabilities. Finally, our approach can generate a diverse set of trajectories between a given set of start and goal locations, which can capture different collision-avoidance behaviors.
Read the full abstract ▾