Robotics · Cross-listed from eess.SY
Feasibility and Singularity in High-Order Safety-Critical Control for Quadrotor UAVs
Omayra Yago Nieto and Leonardo Colombo
We study high-order safety-critical control of quadrotor teams under bounded inputs and pairwise collision-avoidance constraints. Squared-distance barriers may lose thrust effectiveness when the relative displacement is orthogonal to the available thrust directions, while nonsingular constraints may still be jointly infeasible under shared bounds. We characterize both phenomena through pairwise effectiveness and aggregate feasibility measures. A torque-aware dynamic extension exposes attitude torques in a fourth-order barrier and prevents the extended-input row from vanishing under positive thrust. Gaussian processes directly learn the fourth-order HOCBF residual, providing robust margins without differentiating unknown perturbations. Under residual-bound and persistent-feasibility assumptions, the resulting QP guarantees collision avoidance and recovers the nominal input whenever it satisfies the robust safety and actuator constraints.
Read the full abstract ▾
Robotics
Kinematics-Grounded Agentic AI for Robotic Additive Manufacturing Process Planning
Jingzhan Ge, Ruimin Chen, Azadeh Haghighi et al. (+2 more)
Robotic additive manufacturing (AM) extends material-extrusion printing beyond gantry kinematics but makes process planning
robot-dependent. A slicer-generated plan that appears favorable in part coordinates can become infeasible or robotically unfavorable on a manipulator because slicer-process decisions and part orientation determine the generated path, while part orientation and workspace placement affect its kinematic realization. Existing AM tools, large
language model (LLM)-based decision-support methods, and digital-shadow systems do not provide integrated pre-execution evaluation of these coupled decisions. This paper presents agentic robotic additive manufacturing (A-RAM), an
agent-specialist-tool framework that converts user intent and a part file into traceable, execution-ready plans. The LLM interprets manufacturing objectives and constraints, identifies prescribed and searchable planning variables, and encodes this
reasoning in a schema-constrained request; a deterministic Planning
Agent instantiates the corresponding search workflow, while domain tools compute quantitative evidence for slicing, placement, inverse kinematics, trajectory timing, Joint-6 jerk, and extrusion. The framework is evaluated on a six-axis robotic-arm AM
cell through three case studies covering expert-specified planning, goal-only planning, objective-dependent infill screening, and geometry-dependent orientation-placement selection. Across the evaluated candidate sets, selected plans achieve up to 53.5% lower maximum Joint-6 jerk and 48.3% lower mean absolute Joint-6 jerk than the least favorable valid candidates, while objective-specific infill screening yields motion-plan completion times up to 40.1% shorter and extrusion paths up to 12.7% shorter than the corresponding least favorable screened patterns.
Read the full abstract ▾
All authors Jingzhan Ge, Ruimin Chen, Azadeh Haghighi, Jiong Tang, Farhad Imani
Robotics
ViLoMan: Learning Visual-Proprioceptive Whole-Body Loco-Manipulation Skills for Humanoid Robots
Zejie Tian, Ruibing Hou, Bingpeng Ma et al. (+2 more)
Humanoid loco-manipulation requires adaptive whole-body coordination to seamlessly integrate locomotion and physical interaction. Despite recent advances, learning autonomous loco-manipulation remains challenging due to the scarcity of diverse, physically executable
robot-object interaction data and the difficulty of learning unified whole-body control directly from onboard observations. We present ViLoMan, a scalable framework for autonomous humanoid loco-manipulation. ViLoMan first transforms partial kinematic demonstrations of human-object interactions into complete, physically executable
robot trajectories. It then leverages these trajectories within a teacher-student
distillation framework to learn a unified policy that maps egocentric depth observations and proprioceptive measurements directly to joint-level whole-body actions. During deployment, the policy requires neither reference motions nor intermediate commands. We evaluate ViLoMan on door-closing tasks across diverse door configurations and
robot initial conditions in both simulation and the real world. Experimental results demonstrate that a single policy enables a Unitree G1 humanoid to complete the full task using only onboard depth sensing and proprioception, while generalizing robustly across task variations and transferring effectively from simulation to reality. Project page: viloman-anonymous.pages.dev.
Read the full abstract ▾
All authors Zejie Tian, Ruibing Hou, Bingpeng Ma, Börje F. Karlsson, Shiguang Shan
Robotics
Dynamic-LIVO: A Dynamic-Aware LiDAR-Inertial-Visual Odometry System Using Spatio-Temporal Normals
Zhixin Zhang, Samuel Ahiwe, Matthew Hale et al. (+2 more)
This paper proposes Dynamic-LIVO, a dynamic-aware
LiDAR-Inertial-Visual Odometry (LIVO) system for robust state estimation and static colored mapping in dynamic environments. Dynamic-LIVO employs Spatio-Temporal (S-T) normal analysis to identify dynamic
LiDAR points and propagates the resulting classification to both
LiDAR-inertial and visual-inertial updates, preventing dynamic
LiDAR measurements and their associated visual observations from affecting state estimation and mapping. However, S-T normal estimation can be unreliable in newly observed and spatially sparse regions due to insufficient spatio-temporal observations. To address this issue, we introduce a time-delayed S-T normal estimation strategy that defers the classification of insufficiently constrained points and re-evaluates them as additional observations become available. This strategy improves dynamic classification reliability while preserving valid static points for map construction. Extensive experiments on public and self-collected
datasets with diverse sensor configurations demonstrate that Dynamic-LIVO improves localization accuracy and produces cleaner static colored maps in challenging dynamic environments. The source code and self-collected
dataset will be publicly released upon acceptance.
Read the full abstract ▾
All authors Zhixin Zhang, Samuel Ahiwe, Matthew Hale, Liang Zhao, Pawel Ladosz
Robotics
SemSafe-3DGS: Semantic Risk-Aware Active Navigation in Uncertain 3D Gaussian Splatting Maps
Amirhossein Mollaei Khass, Athanasios Cosse and Nader Motee
Autonomous
robots operating in partially observed environments must navigate safely while acquiring observations that improve future planning. Existing safety formulations generally reason primarily about geometry. Consequently, geometrically similar scene elements may induce comparable control responses despite having different semantic consequences. We present a semantic risk aware safe-active perception framework for navigation in attributed 3D Gaussian maps. Semantic attributes modulate an Average Value-at-Risk collision clearance model through class dependent risk weights, allowing safety-critical Gaussian primitives to receive greater influence in the composite barrier. The resulting weighted clearances are aggregated into a control barrier function, while a trajectory-relevant active perception barrier promotes observations that reduce
geometric map uncertainty along the
robot's anticipated motion. Both objectives are integrated in a unified CBF-QP that enforces semantic risk-aware collision avoidance as a hard constraint while relaxing information acquisition when it conflicts with safety or task progress. Experiments demonstrate efficient safety constraint, improved navigation through active perception, semantic dependent trajectory adaptation, and real-
robot execution under Ackermann dynamics.
Read the full abstract ▾
Robotics
A Morphing Aerial Robot With Thruster-Integrated Flexible Continuum Links for Shape Adaptive Aerial Manipulation
Eri Sawada, Kazuki Sugihara, Ayano Miyamichi et al. (+2 more)
In recent years, aerial manipulation has attracted increasing
attention as a key to expand the application of aerial
robots. In this work, we focus on two major research directions for achieving versatile aerial manipulation: (i) acquiring high environmental adaptability using soft manipulators, and (ii) expanding the feasible wrench space by distributing thrusters along the manipulator. However, no aerial
robot has simultaneously satisfied these two requirements. Therefore, in this paper, we propose a morphing rotor-distributed aerial
robot with flexible continuum links that achieves both high shape adaptability and an expanded wrench space. The flexible continuum links function as soft manipulators, passively conforming to the shape of the environment, while the thrusters distributed along the continuum links expand the feasible thrust wrench space and enable the end-effector to exert large interaction forces. To realize the proposed
robot, it is essential to suppress vibrations of the lightweight continuum links. Thus, we develop a composite leaf-spring structure that provides both high torsional and vertical stiffness, and vibration-suppressing control methods. Using these implementations, we demonstrate stable flight and a variety of aerial manipulation tasks. To the best of our knowledge, this is the first work to realize aerial manipulations using flexible links with an integrated thruster.
Read the full abstract ▾
All authors Eri Sawada, Kazuki Sugihara, Ayano Miyamichi, Kunio Kojima, Kei Okada
Robotics
GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning
Ruiyang Wang, Hao-Lun Hsu, Swarajh Mehta et al. (+3 more)
Large
language models (LLMs) provide a flexible interface for long-horizon
robot planning, but generated plans often fail to respect embodiment constraints, recover from planning errors, or reason effectively under partial observability. We present GAVEL, a framework for verifying and repairing long-horizon LLM planning built around an explicit graph world model. The graph represents relevant object-relations, action pre-conditions and effects, and probabilistic beliefs over unobserved object locations. This model can predict the consequences of LLM-generated actions before execution, detect violations, and repair those whose corrections follow directly from the world model. This method also reserves LLM replanning solely for errors requiring semantic
reasoning. For multi-task instructions, GAVEL reasons over distributions of possible object locations to reorder remaining subtasks and minimize expected search cost. We evaluate GAVEL on BEHAVIOR-1K across 100 single long-horizon tasks and 500 multi-task instructions. With Qwen3-8B, GAVEL improves single-task success from 41.2% to 91.8% and multi-task success from 19.9% to 92.6%. Distributional belief
reasoning also reduces travel distance by approximately 5.4% compared with a static variant. These improvements show that an explicit graph world model harness can substantially improve the reliability and efficiency of long-horizon embodied planning across compact and frontier hosted LLM capabilities.
Read the full abstract ▾
All authors Ruiyang Wang, Hao-Lun Hsu, Swarajh Mehta, Jiwoo Kim, Zhihao Dou, Miroslav Pajic
Robotics
OHRID-Retail: An Open Multimodal Dataset of Human Activity in Retail Environments
Xiangrui Wang, Yuetong Wu, Jalen Beeman et al. (+6 more)
Open
datasets describing human behavior in environments shared with mobile
robots remain limited, particularly for retail activities that combine locomotion, reaching, object handling, and
robot guided movement. This paper introduces OHRID Retail, an open, human centered
multimodal dataset collected from 16 healthy adults performing a simulated shelf picking task under three within participant conditions: no
robot, low speed
robot guidance, and high speed
robot guidance. Each participant completed two trials per condition. Whole body kinematics were recorded using 17 Xsens Awinda inertial sensors and muscle activity was measured at 10 locations using Delsys Trigno surface electromyography sensors. Descriptive analyses demonstrate variation in whole body movement intensity and muscle activation across
robot interaction conditions and body locations. OHRID Retail provides openly available raw recordings, processed measures, documentation, and reproducible analysis resources. The
dataset can support research in human activity recognition,
multimodal sensor fusion, occupational biomechanics, ergonomics, human aware
robot navigation, and human
robot interaction in retail and related shared environments.
Read the full abstract ▾
All authors Xiangrui Wang, Yuetong Wu, Jalen Beeman, Robert Cook, Yu Gu, Nathanial Pearson, Trevor Smith, Read Hayes, Boyi Hu
Robotics
Learning Safe Humanoid Navigation from Reduced Order Models
William D. Compton, Zachary Olkin, Ryan Bena et al. (+1 more)
Research in humanoid robotics has achieved rapid progress in locomotion, and recent results have pushed the boundary on autonomous navigation. We demonstrate that a standard single-stage RL navigation pipeline struggles to scale to multi-level and multi-story terrain, limited by the difficulty of complex humanoid terrain interactions such as stairs. To overcome this challenge, we decompose the navigation problem into two pieces. First, we train a policy operating on the reduced order dynamics but with full 3D
LiDAR observations to navigate complex, multi-story terrain. We then utilize this navigation knowledge to kickstart a policy operating on the full-order humanoid dynamics, with a frozen locomotion policy in the loop. Additionally, we demonstrate that applying a Poisson safety filter to the navigation policy output recovers safety in the presence of out-of-distribution obstacles, without dropping navigation success rate. We demonstrate the resulting RoM-Nav policy on a Unitree G1, accomplishing mapless multi-floor navigation covering trials with over 10m of vertical displacement and over 100m of path length. Project page with
videos https://wdc3iii.github.io/rom-nav/ .
Read the full abstract ▾
All authors William D. Compton, Zachary Olkin, Ryan Bena, Aaron D. Ames
Robotics
Grasping by interconnection: robust closing motions from coarse object templates
Julien Vanderheyden, Guillaume Drion, Fulvio Forni et al. (+1 more)
Dexterous
robot hands must often grasp objects whose shape, size, and pose are known only approximately. Grasp planners typically require accurate object models or correct errors with feedback, but how much inaccuracy a closing motion can tolerate on its own remains unclear. To address this question, we designed a motion planner based on four principles: a coarse template of the object, human grasp types, an object-centric interaction, and compliant, sliding contacts instead of prescribed contact points. This paper presents the planner, implemented through virtual model control, and its evaluation on a Shadow Dexterous Hand. Without feedback, the planned closing motions tolerated size errors of about 1cm and pose errors of several centimeters and tens of degrees, a wider range than a state-of-the-art data-driven planner in 25 of 27 tested conditions. They also grasped 82.5% of 80 everyday objects and succeeded within an autonomous pipeline. Robustness can thus be designed into the closing motion itself, rather than left only to feedback. This planner opens a path toward reliable manipulation in uncertain settings, which we will pursue by combining it with adaptive feedback control on the physical hand.
Read the full abstract ▾
All authors Julien Vanderheyden, Guillaume Drion, Fulvio Forni, Pierre Sacré
Robotics
4D Radar Perception Algorithms for Autonomous Driving: A Review
Xumin Wu, Jun Zhou, Jilin Mei et al. (+2 more)
Research on 4D millimeter-wave radar perception
algorithms has flourished in recent years, extending from signal processing and object detection to semantic segmentation, motion estimation, occupancy prediction, and dynamic scene reconstruction. This review organizes the field according to the
evolution of perception tasks and
algorithms. It first introduces radar fundamentals, data representations, and quality-enhancement methods, and then reviews object-level perception, motion and localization, local and dense spatial perception, and dynamic scene understanding. Across these directions, we compare radar-only learning,
multimodal fusion, and cross-modal supervision and knowledge
distillation. Particular
attention is paid to how elevation, Doppler measurements, and radar physical priors are exploited across tasks. We further summarize the task coverage, input data, annotations, and evaluation protocols of existing
datasets, clarifying the empirical support for different research directions. Finally, we discuss the common challenges and future directions of 4D radar perception for autonomous driving. This review provides a task-oriented perspective on the transition from sparse object perception to dynamic spatial understanding.
Read the full abstract ▾
All authors Xumin Wu, Jun Zhou, Jilin Mei, Chen Min, Yu Hu
Robotics
REACT: A Fully Spiking State-Space Model for Real-Time Event-Driven Temporal Perception
Geoffroy Keime, Nicolas Cuperlier and Benoit R. Cottereau
Robotic systems operating in dynamic environments require visual perception that evolves continuously with the incoming sensory stream. Event cameras provide microsecond temporal resolution and asynchronous sensing, but most learning-based methods accumulate events into frames or temporal bins, introducing an integration delay that can limit fast reaction. Here we propose REACT, a fully spiking state-space model for event-driven temporal perception that processes raw events one by one, without temporal accumulation. REACT uses a complex-valued spiking
neuron, C-SiLIF, whose continuous-time dynamics are driven by the physical inter-event interval, allowing its internal state to evolve at the temporal resolution of individual events. We evaluate REACT on gesture recognition and time-to-collision (TTC) estimation from full-field event streams, without a target bounding box or localization input. On EvTTC, REACT achieves a 9.59% relative TTC error with 4.6 ms end-to-end inference latency, within 0.15 percentage points of the best learned method while requiring no target prior. At the
dataset's mean approach speed, this latency corresponds to only 4 cm of vehicle motion, compared with 1 m for the fastest competing learned method. REACT further supports anytime TTC prediction, zero-shot transfer to a different driving sequence, and INT8 quantization, reducing the estimated energy consumption from 18.5 to 2.8 mJ per 32,768 events. These results show that event-driven spiking state-space dynamics can provide low-latency, continuously updated temporal perception for reactive robotic systems.
Read the full abstract ▾
Robotics
ULOHA: An Underwater Bimanual Robot System for Robot Learning
Masato Kobayashi and Takeru Tsunoori
Underwater visuomotor policy learning has focused primarily on single manipulators, while bimanual imitation learning has been studied largely in air. We present ULOHA, an underwater bimanual
robot learning platform that combines custom-designed leader–follower hardware with software extensions to LeRobot, integrating teleoperation, multi-view sensing, demonstration collection, policy training, and autonomous deployment. Real-
robot experiments demonstrate a range of coordinated underwater bimanual behaviors, including inter-arm transfer, shared-object manipulation, and buoyancy-driven interception. We evaluate ACT,
Diffusion Policy, and the
vision–
language–action model SmolVLA on the platform. We investigate how learning methods and execution strategies developed for manipulation in air perform underwater, examining bubble disturbances, buoyancy-driven object motion, action-execution horizons, and real-time chunking. A separate single-arm study examines policy transfer between air and water and shows that demonstrations spanning both media support execution in both under the tested conditions. ULOHA provides a unified experimental platform for studying underwater bimanual
robot learning under the coupled perceptual and physical effects of underwater environments. Additional material: https://mertcookimg.github.io/uloha/
Read the full abstract ▾
Robotics
DITTO: Dexterous Interface for Transparent TeleOperation
Joaquin Palacios, Katelyn Lee, Cheng Zhang et al. (+2 more)
Collecting data for manipulation with high-DOF hands is challenging, as interfaces must capture rich hand motion while rendering the contact interactions essential for precise manipulation. Existing data collection approaches face a trade-off: teleoperation ensures deployment consistency but lacks force feedback, while handheld (in-the-wild) systems provide natural force transparency but introduce a visual embodiment gap at deployment. We present DITTO, a Dexterous Interface for Transparent TeleOperation, which resolves this through the anatomically informed co-design of a dexterous 7-DOF robotic hand and a kinematically equivalent motorized exoskeleton. A 1-to-1 actuator mapping between the exoskeleton and robotic hand enables handheld (in-the-wild) data collection and bilateral teleoperation with joint-level force feedback unified in a single platform. We demonstrate that the DITTO exoskeleton spans the operator's natural index-to-thumb workspace, and showcase DITTO's dexterous capabilities via learned policies on contact-rich tasks.
Read the full abstract ▾
All authors Joaquin Palacios, Katelyn Lee, Cheng Zhang, Zhanpeng He, Matei Ciocarlie
Robotics · Cross-listed from cs.CV
PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics
Bardienus P. Duisterhof, Kaifeng Zhang, Adam Hung et al. (+5 more)
World models endow perceptual systems with the ability to predict how scenes evolve under interaction. They are most beneficial when trained on diverse volumes of data, to instill a rich prior into downstream applications. Existing methods typically require
robot action labels to learn action-conditioned 3D dynamics, which excludes web
video data from the training pool. We study 3D point track completion as a pre-training objective for learning transferable 3D dynamics without
robot data. Given a single RGB-D observation and sparse partial 3D trajectories (tracks), we predict future 3D tracks of all observed points. We show this objective produces a rich 3D dynamics prior, without requiring
robot action labels. We contribute a diverse
dataset of 2.9 million synthetic frames spanning deformable, articulated, and rigid objects, and use it to train PointZero. We show that a flexible and expressive
transformer, PointZero, outperforms prior methods on the same data. We demonstrate the utility of our pre-training objective by post-training PointZero for two downstream applications: (1) action-conditioned 3D dynamics prediction and (2) imitation learning. When fine-tuned to condition on end-effector pose, PointZero outperforms the baselines on the recent PGND 3D dynamics
benchmark. When fine-tuned to predict
robot actions and 3D tracks, PointZero outperforms or matches the baselines on 6/7 simulated and real-world
robot manipulation tasks. We furthermore evaluate training PointZero from scratch to isolate the benefits of our proposed architecture from those of our proposed pre-training objective and
dataset. We release the
dataset, checkpoints, and full training recipe.
Read the full abstract ▾
All authors Bardienus P. Duisterhof, Kaifeng Zhang, Adam Hung, Bowen Wen, Stan Birchfield, Yunzhu Li, Deva Ramanan, Jeffrey Ichnowski
Robotics · Cross-listed from cs.CV
In-Context Robot Learning with VLM Agents
Dongzhou Cheng, Taoran Yi, Ye Fang et al. (+12 more)
Enabling
robots to adapt to unfamiliar environments as readily as humans remains a moonshot goal of embodied AI. No finite collection of demonstrations can cover every task and situation a
robot will encounter, making the ability to learn from context at deployment essential for generalization. Such in-context learning (ICL), however, remains largely beyond the reach of existing robotic policies. The broad agentic capabilities of commercial
vision-
language models (VLMs), such as GPT-6 Astra, raise a compelling question: can these models learn from demonstrations, examples, and interaction feedback, then translate that information into executable and verifiable
robot behavior from a new initial state without gradient updates or persistent changes to task-specific parameters? We introduce GPT-Policy, a general-
agent framework for in-context
robot learning. GPT-Policy integrates a context compiler that preserves task-relevant visual transitions, a VLM that proposes
robot-tool actions, and a constrained controller that verifies and executes each action and reports its outcome. We evaluate its reliability and limitations through task success and efficiency metrics, matched comparisons across models, and controlled context ablations. In real-
robot trials, human
video demonstrations improve task completion even without
robot action labels, while aligned action references yield further gains on contact-sensitive tasks. These findings position GPT-Policy as a step toward
robot adaptation through in-context learning, providing an empirical foundation for translating the general-purpose capabilities of VLMs into physical behavior and clarifying the challenges that must be overcome for reliable deployment.
Read the full abstract ▾
All authors Dongzhou Cheng, Taoran Yi, Ye Fang, Xingwu Zhang, Fan Feng, Yixuan Li, Gengxiong Zhuang, Rongze Wang, Shuai Yang, Wei Song, Weizhi Xue, Minyan Wu, Jie Gui, Jiaqi Wang, Tong Wu
Robotics
Dreaming the Sound of Contact: Leveraging Video and Audio Generation for Zero-Shot Force-Aware Manipulation and Data Generation
Guanhua Ji, Tianyu Li, Dayoon Suh et al. (+3 more)
Video generation models have advanced rapidly and can now synthesize plausible
videos of
robot manipulation from image and text
prompts. Recent work extracts
robot actions directly from such generated
videos, but the result is purely kinematic and lacks force information, causing failures in contact-rich tasks where appropriate contact forces are essential for success. We present a pipeline that jointly leverages generated
video and audio to derive motion trajectories and desired-force profiles. The force profile is shaped by the loudness of the generated contact sound, and we execute the resulting force-aware trajectories on a Franka
robot using a closed-loop force regulator. We evaluate our pipeline on multiple tasks that require making contact and demonstrate successful zero-shot manipulation where a kinematic-only baseline fails. We also show that the pipeline can be used as a data generation engine to train policies that achieve the tasks in a closed-loop manner. Project website,
videos, and
dataset: https://dreamingcontactsound.github.io/
Read the full abstract ▾
All authors Guanhua Ji, Tianyu Li, Dayoon Suh, Yuqian Zhang, Boyan Zhang, Nadia Figueroa
Robotics
rMuscle: Robotic Muscle Memory for Efficient Vision-Language-Action Model Inference
Kaijun Zhou, Zhiyang Li, Le Chen et al. (+1 more)
Factory work is a promising early scenario for embodied AI: assigning repetitive manual jobs to
robots has clear economic payoff, and a structured station keeps the jobs tractable for current policies.
Vision-
Language-Action (VLA) models now dominate as the policy paradigm for these
robots. The inference latency of VLA models directly affects
robot responsiveness and motion smoothness. However, existing VLA inference frameworks do not fully exploit the characteristics of embodied workloads or account for the distinct bottlenecks across different stages of VLA inference. In this paper, we first characterize embodied workloads and identify substantial task similarity across repeated
robot executions. We further find that such similarity extends beyond observations and action trajectories to internal model states. Drawing on these observations, we present rMuscle, a real-time VLA inference framework inspired by human muscle
memory. It exploits cross-execution similarity through a dual-phase muscle-
memory cache. The Context Cache reuses visual-
token outputs to reduce computation, while the Action Cache reuses
neuron activation patterns to reduce weight accesses. We keep both the cache
memory footprint and access overhead low through online cache recomputation, sliding-window cache
retrieval, and mask sharing across consecutive denoising steps. rMuscle achieves 1.29-1.42X speedup on RTX 4090 and Jetson Thor across LIBERO, RoboTwin, and physical manipulation tasks, while maintaining the original success rates on real-world
robots.
Read the full abstract ▾
All authors Kaijun Zhou, Zhiyang Li, Le Chen, Jinyu Gu
Robotics
ElastiQP: An Always-Feasible QP Solver for Constrained Robot Control
Daniel Morton, Jon Arrizabalaga, Zachary Manchester et al. (+1 more)
As
robot capabilities increase, quadratic programming (QP)-based controllers must account for a similarly increasing number of constraints to ensure safe, reliable operation. Yet, with each added constraint, this introduces more chances of momentary conflict: in which case, a QP solver that returns an "infeasible" status leaves the controller with nothing to execute. To address this, we introduce ElastiQP, a modified dual active-set QP solver that relaxes every inequality constraint with an exact, per-constraint l1 penalty while keeping equality constraints (dynamics) hard. Notably, ElastiQP does so by folding the slack variables into the solver analytically, maintaining a constant size of the condensed linear system. On a suite of
robot control
benchmarks, ElastiQP achieves microsecond-level performance, matching or outperforming leading modern solvers on feasible problems. On infeasible problems, ElastiQP handles these gracefully, confining violations to strictly the conflicting inequality terms, returning a usable solution up to 40x faster than the best alternative solvers. ElastiQP is available as an open-source C++ header-only library, with Python and JAX interfaces, at https://github.com/StanfordASL/elastiqp.
Read the full abstract ▾
All authors Daniel Morton, Jon Arrizabalaga, Zachary Manchester, Marco Pavone
Robotics
"What's going to happen after I'm gone?": Parent Perspectives on Technology in Supporting Independent Living for Adults with Intellectual Disabilities
Alexander Tyshka, Andrea Macklem-Zabel, Absalat Getachew et al. (+2 more)
Adults with intellectual and developmental disabilities (IDD) are increasingly transitioning from family homes towards semi-inde\-pendent living. As parents hand off the role of primary caregiver, they face numerous challenges in arranging consistent and quality support. Our research centers on understanding these caregiving routines. By focusing on the unique lived experience of parents, who possess extensive explicit and tacit knowledge of their adult child's requirements, we aim to map the management of care they provide. This foundational understanding is essential to identifying how assistive technologies can effectively serve a key role in supporting adults with IDD in this transition. In this work, we interviewed 16 parents of adults with IDD beginning this transition to understand: 1) how they currently provide support for daily living and what makes their support effective; 2) what their experiences and perceptions are regarding the use of technology; and 3) how they envision assistive technology successfully integrating with their adult child's new home or with other supports to promote independence. Our thematic analysis produced four themes: common modes of support, inside the routine to support growth, the fragility of continuity of care across transitions, and participants' perceptions and experiences of assistive technology. Building on these findings, we derive four design principles for growth-oriented assistive technology. These include pre-transition onboarding to capture caregiver tacit knowledge; structured scaffolding towards long-term growth; adaptive sensing that responds to day-to-day variability; and customization for the individual balanced with consistency for the care network.
Read the full abstract ▾
All authors Alexander Tyshka, Andrea Macklem-Zabel, Absalat Getachew, Foong Ling Chen, Wing-Yue Geoffrey Louie
Robotics
Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator
Zhongyu Chen, Yuxuan Nai, Qian Chen et al. (+8 more)
Bipedal loco-manipulation enables
robots to interact with objects beyond the nominal workspace of their arms by coordinating locomotion and manipulation. Realizing this capability requires a low-level whole-body controller that translates task-level manipulation goals into coordinated arm and leg motions while maintaining balance. We present a unified whole-body controller trained with reinforcement learning that directly maps 6-DoF end-effector targets to coordinated actions for the bipedal base and robotic arm. Given only an end-effector target, the learned controller autonomously coordinates reaching, postural adaptation, and stepping without explicit base-velocity or footstep commands. A reward-gating strategy regulates the trade-offs among end-effector tracking, locomotion, and balance during training, while a temporal context
estimator combines windowed
Transformer encoding, recurrent GRU
memory, and auxiliary dynamics prediction to extract dynamics-relevant information from observation history. Real-
robot experiments demonstrate that the same controller supports reaching, postural adaptation, and stepping under commands from VR teleoperation, a learned
diffusion policy, and scripted trajectories, providing a common end-effector interface for diverse manipulation tasks.
Read the full abstract ▾
All authors Zhongyu Chen, Yuxuan Nai, Qian Chen, Yidong Zhu, Chen Jing, Qihan Wang, Xudong Li, Zhizhan Li, Leixin Chang, Liangjing Yang, Hua Chen
Robotics
CaSCo: Cascade-Aware Soft-Collision Motion Planning
Shivaram Kumar, Gaoyuan Liu and Yoonchang Sung
Conventional motion planning treats collision as a binary constraint, although contact with different objects can have drastically different consequences. A
robot may safely brush against a cardboard box while even minor contact with a glass, laptop, or unstable object may be undesirable. Moreover, a direct
robot–object collision can move the contacted object and trigger secondary object–object collisions, making the risk of a motion depend on the physical
evolution of the scene rather than only on the
robot's
geometric path. We present CaSCo, a cascade-aware soft-collision motion planning framework in which a
vision-
language or
language model assigns semantic risk to objects and a physics simulator predicts the consequences of candidate
robot motions. CaSCo searches for a path that minimizes the total semantic risk of the unique objects displaced either directly by the
robot or indirectly through cascaded collisions. Because collisions change the environment, we augment roadmap states with the predicted object arrangement and the set of objects whose risk has already been incurred. We develop an optimal graph-search
algorithm with an admissible and consistent cascade-relaxed heuristic and caching and
pruning mechanisms for efficient search. Experiments in cluttered manipulation environments evaluate semantic risk, cascade
reasoning, planning efficiency, and real-
robot operation.
Read the full abstract ▾
Robotics
Examining the Difference in Human Behavior Between Virtual and Real-World Human-Robot Teaming
Sean Dallas, Absalat Getachew, Motaz AbuHijleh et al. (+4 more)
Prototyping and evaluating human-
robot teaming (HRT) scenarios in the real-world is costly. Virtual simulation of HRT scenarios has been adopted as an alternative to conducting user studies in the real-world to investigate user perceptions, behaviors, and performance during human-
robot interactions. The consistency of human behavior between the real and virtual-worlds is integral to the validity of utilizing such virtual experimentation. This paper presents a user study examining the difference in human behavior between an HRT scenario conducted in a virtual vs real environment. We employed mixed-methods to examine team performance and human factors during the HRT. Our quantitative results showed a significant difference in workload between the two modalities. Our qualitative analysis expanded on the quantitative results and found differences in participants' strategy, their mental model of
robots, and the type of trust they had for
robots between modalities.
Read the full abstract ▾
All authors Sean Dallas, Absalat Getachew, Motaz AbuHijleh, Andrea Macklem-Zabel, Douglas Zytko, Mark Brudnak, Wing-Yue Geoffrey Louie
Robotics
SOL-SLAM: Inverse Compositional Gauss-Newton Direct Registration for Fast Sonar-Only Local SLAM
Kalvik Jakkala and Jason O'Kane
Autonomous underwater navigation typically relies on complex and expensive multi-modal sensor suites designed to prioritize global Simultaneous Localization and Mapping (SLAM) accuracy. However, local reactive behaviors such as coarse navigation and obstacle avoidance require only local consistency—a capability that should be feasible using only a Forward-Looking Sonar (FLS), yet remains largely unaddressed, leaving a critical gap in FLS-only local SLAM. Moreover, existing acoustic SLAM frameworks predominantly rely on sparse feature extraction methods that discard substantial portions of the already information-sparse acoustic returns. To overcome these limitations, this work introduces a dense direct registration approach that aligns full acoustic intensity scans to a recursively updated local map. Real-time execution is achieved via an Inverse Compositional Gauss-Newton optimization strategy that minimizes computational overhead. Experimental evaluations show that this dense method yields significant improvements on translation error compared to sparse keypoint baselines, maintaining stable sub-meter tracking precision over wide displacement gaps. Moreover, this approach delivers odometry performance comparable to multi-sensor fusion pipelines (FLS, DVL, and IMU), bypassing expensive payload dependencies in feature-rich environments. We validate real-world applicability through AUV field trials, running the full local SLAM approach onboard an embedded, resource-constrained computer.
Read the full abstract ▾
Robotics
Body-Motion Control of a Simulated Aerial Swarm from a First-Person View
Yang Chen, Darius Giannoli and Dario Floreano
First-person-view (FPV) teleoperation of aerial swarms requires an operator to coordinate collective translation, viewing direction, and formation spacing. We present an upper-body interface that maps torso inclination, hand position, and head rotation to five continuous command dimensions. Neutral postures and motion ranges are calibrated for each participant. In a within-subject study, 14 participants navigated a simulated 15-
agent swarm through three-dimensional obstacle courses using this interface and a conventional transmitter. Body-motion control reduced completion time by 19.4% and centroid path length by 7.0%, and increased path directness. Delivered-command variation was 88.8% lower, and concurrent command changes were more frequent. These command measures characterize the complete interfaces, which differed in calibration and filtering. No differences were detected in gate-centering error, collection yield, crash or disconnection counts, overall workload, or usability. All participants reported higher physical demand with body-motion control. The implemented interface therefore improved FPV navigation efficiency at the cost of greater physical demand.
Read the full abstract ▾
Robotics
KINO: A Keyframe Interface for VLM Planning and Whole-Body Control in Humanoid Loco-Manipulation
Sitong Chen, Fatemeh Zargarbashi, Jin Cheng et al. (+2 more)
Humanoid loco-manipulation requires
robots to interpret task instructions and scene semantics while executing coordinated whole-body motions. We propose a hierarchical framework that uses motion keyframes as an intermediate representation between
Vision-
Language Model (VLM) planning and Reinforcement Learning (RL) control. Each keyframe specifies a target whole-body
robot pose and, when applicable, an object pose. Given a
language instruction, scene observations, and execution feedback, the VLM selects successive task-relevant keyframes from a predefined library. The selected keyframes are retargeted to the current scene to account for object poses and dimensions. A keyframe-conditioned whole-body policy then generates joint-level actions to reach these goals. We introduce a saliency-based keyframe sampling strategy for low-level policy training that improves end-to-end task success rate from 44% to 92% when using sparse VLM keyframes. We evaluate our framework on object pickup, transport, and placement tasks in simulation and on a Unitree G1 humanoid. The system successfully performs both one- and two-handed manipulation and generalises to placement locations beyond the training reference data.
Read the full abstract ▾
All authors Sitong Chen, Fatemeh Zargarbashi, Jin Cheng, Tianxu An, Stelian Coros
Robotics · Cross-listed from eess.SY
Towards Interaction Regulation from Human Feedback via Free Energy Minimization
Maria Paula Diaz Monfort, Cinzia Tomaselli, Michael Richardson et al. (+1 more)
A central challenge across control and learning is the design of mechanisms regulating the interactions between humans and autonomous
agents. Inspired by the free energy principle from computational neuroscience, we introduce a control-theoretical framework to integrate human preferences online into an
agent policy. We turn the framework into an open control architecture and validate our approach using a human-in-the-loop experimental testbed involving a rover navigating via onboard sensing. The human, remotely located and equipped with virtual reality headsets, shares the same sensory information as the rover. Human preferences are provided to the rover via gestures which introduce both cooperative and competitive interactions between the
agent goal and the preferences. The experiments show that interactions are regulated, validating the proposed approach.
Read the full abstract ▾
All authors Maria Paula Diaz Monfort, Cinzia Tomaselli, Michael Richardson, Giovanni Russo
Robotics
SEAM: Submap-Anchored Evidence for Lifelong LiDAR Mapping under Trajectory Deformation
Kyuwon Kim
We propose SEAM, a
LiDAR-based lifelong mapping framework. Instead of relying on a single anchor spanning the entire session, SEAM generates evidence based on a trajectory optimized with submap-level anchors, and performs dynamic object removal and change detection. Through submap-level reprojection, the generated evidence remains usable even if the trajectory is subsequently modified by a new session, eliminating the need to recompute the entire process from scratch. SEAM suppresses geometrically unreliable inter-session loop edges using a DOP-based confidence measure. Suppressing unreliable loop edges prevents
alignment errors. SEAM also uses a directional voxel-wise evidence model. The model accounts for occupancy patterns that vary with ray direction. Direction-aware evidence separates dynamic objects from environmental changes more precisely. Experiments on a real construction-site
dataset and a long-term multi-session
dataset show that SEAM achieves higher accuracy and faster processing than existing methods.
Read the full abstract ▾
Robotics
Asymptotically Optimal Multi-Robot Task and Motion Planning
Thi Thuy Ngan Duong, Cheuk Tung Shadow Yiu, Rahul Shome et al. (+1 more)
Multi-
robot task and motion planning (MR-TAMP) requires jointly
reasoning about discrete task decisions and continuous collision-free motions of multiple interacting
robots. Although asymptotically optimal
algorithms have been developed for task and motion planning, extending these guarantees to the multi-
robot setting introduces an important challenge: different task transitions may involve different subsets of
robots and therefore impose constraints of different dimensions on the composite configuration space. Consequently, an asymptotically optimal planner must not only optimize motion within each task mode, but also ensure sufficient exploration of the different types of transitions connecting them. We characterize this transition structure and establish sufficient conditions for global asymptotic optimality in MR-TAMP, requiring persistent coverage of relevant transitions and asymptotically improving motion planning within connected feasible regions. Based on these conditions, we develop an efficient asymptotically optimal MR-TAMP
algorithm that combines evolving individual-
robot roadmaps with implicit tensor-product search, avoiding explicit construction of the composite roadmap. The planner further employs conditional transition sampling, lazy collision checking, and mode- and solution-level guidance to improve finite-time planning efficiency while retaining persistent exploration. The resulting framework provides asymptotic optimality guarantees for multi-
robot manipulation while efficiently exploiting the structure of individual-
robot motion planning.
Read the full abstract ▾
All authors Thi Thuy Ngan Duong, Cheuk Tung Shadow Yiu, Rahul Shome, Yoonchang Sung
Robotics
AdaGeoVLN: Selective Geometry Across Representation Depth and Navigation Time for Vision-Language Navigation
Quan-Dung Pham, Anh Dao, Danh Vinh Le et al. (+7 more)
Vision-
language navigation requires aligning
language with visual observations while maintaining spatial understanding over time. Geometry foundation models (GFMs) expose intermediate representations throughout their hierarchy, but how navigation policies should use these features and retain historical
geometric evidence remains unresolved. We introduce \method{}, a streaming VLN framework that addresses these questions across representation depth and navigation time. Hierarchical GFM–VLM fusion couples earlier, intermediate, and later GFM representations to successive policy stages instead of repeatedly injecting a terminal feature. Navigation-aware GFM
memory retains historical VGGT global-
attention KV states according to instruction relevance,
geometric confidence, and transition novelty under a bounded per-layer budget. Retained states provide
geometric context for subsequent observations before fusion with the policy. Across R2R-CE and RxR-CE, \method{} achieves strong performance using a single RGB stream without additional navigation-specific external data. Controlled ablations show that multi-depth coupling substantially outperforms repeated terminal-feature injection at matched fusion locations. Bounded navigation-aware retention preserves navigation performance while considerably reducing GFM-KV
memory relative to larger-
memory temporal retention. These findings support jointly examining the
geometric representations exposed to the policy and the historical evidence retained for future inference. Code will be released upon acceptance at https://humanoid-research.github.io/adageovln/.
Read the full abstract ▾
All authors Quan-Dung Pham, Anh Dao, Danh Vinh Le, Nguyen Viet Tri Pham, The-Anh Nguyen, Zhirui Dai, Yiyu Chen, Tuyen P. Le, Truong Nguyen, Quan Nguyen