Robotics
Light-Loco-Parkour: Versatile Perceptive Whole-Body Locomotion via Multi-Skill Distillation
Hongming Chen, Zhuoran Li, Hongxi Wang et al. (+9 more)
Existing
humanoid whole-body control systems still fall short of the way humans move through cluttered terrain: they either track expressive whole-body references without terrain generalization, or react to terrain online while leaving the arms, torso, and knees largely unused. We present Light-Loco-Parkour (LLP), an end-to-end perceptive whole-body locomotion system that closes this gap with a single deployable policy. Conditioned only on onboard depth and a velocity command, the policy decides when to walk, balance, climb, step down, or vault, with no reference input, skill label, hand-coded gate, or runtime motion graph. Compared with prior
humanoid systems, LLP makes three contributions. First, it introduces a whole-body perceptive-control pipeline that extends an RL-trained, velocity-tracking locomotion policy with parkour skills learned from object-interacting motions, so the same policy tracks velocity in open terrain, executes whole-body traversal at obstacles, and resumes locomotion afterward. Second, it acquires terrain-conditioned skills from sparse seeds by expanding a single motion into dynamically feasible, terrain-paired references across obstacle geometry, rather than relying on a large motion corpus. Third, it learns autonomous skill transitions from reward, letting the policy decide when and which whole-body skill to invoke from depth and command alone, with no one-hot skill label, hand-coded state machine, or runtime motion generator. Simulation and real-world experiments show high success across both benchmarked terrains and unseen obstacle variations, and the same policy transfers zero-shot to indoor and outdoor hardware experiments. These results demonstrate autonomous perceptive whole-body locomotion on a
humanoid in outdoor settings, using only onboard sensing and a single deployable policy.
Read the full abstract ▾
All authors Hongming Chen, Zhuoran Li, Hongxi Wang, Jiangpeng Hu, Ziliang Li, Peize Liu, QingRui Zhao, Xuhao Liu, Liang Pan, Ximin Lyu, Yuntao Ma, Tingxiang Fan
Robotics
Bicycle Acrobatics with Reinforcement Learning
Shamel Fahmi, Arianna Ilvonen, Samuel Zapolsky et al. (+8 more)
Bicycle
robots are fast and energy efficient, but their simple mechanical design and their underactuated and non-holonomic dynamics make highly agile maneuvers difficult to achieve. Here, we use Reinforcement Learning (RL) to enable a bicycle
robot to learn and compose a diverse repertoire of dynamic acrobatic stunts. Using different RL formulations such as waypoint following, pose reaching, twist tracking, guided tracking, and motion imitation, the
robot acquires autonomous single and multi-table forward and lateral jumps, steerable jumps, front flips, kip-ups, kip-downs, driving, wheelies, bunny hops, and three-point turns. To coordinate these behaviors, we introduce an orchestrator that transitions between policies using state-dependent triggers, enabling robust long-horizon acrobatic stunts. We validate the approach on the Ultra Mobility Vehicle (UMV), a custom bicycle
robot, in simulation and hardware. The
robot repeatedly traverses tables up to 1 m high, performs more than 15 consecutive autonomous jumps while following waypoints, handles previously unseen multi-table configurations, executes continuous repertoires of kipups, jumps, flips, kip-downs, over more than 20 consecutive trials, and performs more than 10 consecutive autonomous and steerable repertoires of wheelies, lateral jumps, and single-wheel jump downs. These results demonstrate that RL can endow bicycle
robots with levels of agility previously associated primarily with legged platforms while preserving the speed and efficiency of wheeled locomotion, establishing a foundation for bicycle acrobatics.
Read the full abstract ▾
All authors Shamel Fahmi, Arianna Ilvonen, Samuel Zapolsky, Yu-Ming Chen, Ravi Boggavarapu, Paul Drews, Ashwin Khadke, Dean Molinaro, Kaiyu Zheng, Alfred Rizzi, Gabriel Nelson
Robotics
Minute-Scale Training for Microrobot Navigation
Yinghan Sun, Aoji Zhu, Xiang Ji et al. (+6 more)
Microrobots hold significant potential for various applications, where targeted
navigation is a basic requirement. Deep reinforcement learning (DRL) has recently emerged as a powerful paradigm for fully autonomous microrobot
navigation. Yet, current DRL-based approaches pay limited
attention to learning efficiency and effectiveness, requiring hours to days for model training. Consequently, this impedes both rapid practical deployment and parameter optimization. To address these challenges, we present a learning framework that enables effective microrobot
navigation policies to be trained within minutes. In the proposed framework, we develop a fully vectorized simulator with more than 10,000 artificial vascular environments, parallelizing dynamics,
LiDAR-inspired perception, and feasibility checks across thousands of environments to achieve roughly 190,000 transitions per second. To achieve effectiveness in fast training, we propose a task-shaping-regularization (TSR) reward framework. The TSR framework accelerates convergence, improves final performance, reduces action variation by at least 33.7%, and increases obstacle clearance by at least 2.1% across all evaluated scenarios. Results show that the proposed learning framework reduces training time to under 10 minutes, while supporting zero-shot deployment across distinct microrobot types and
navigation scenarios. Collectively, this framework can substantially shorten the design loop and accelerate the deployment of autonomous microrobots.
Read the full abstract ▾
All authors Yinghan Sun, Aoji Zhu, Xiang Ji, Yamei Li, Jiachi Zhao, Yun Wang, Li Zhang, Huijun Gao, Lidong Yang
Robotics
GeminiPainter's sequence-formed pipeline comprised of perception, cognition, planning, and action stages
Miguel Altamirano Cabrera, Aleksey Fedoseev, Iana Zhura et al. (+1 more)
We present an autonomous robotic portrait-generation system combining real-time face detection, AI-based sketch generation, and robotic drawing. The system captures
video frames, extracts facial regions, converts them into minimalist single-line sketches using the Gemini
Vision API, optimizes stroke order through graph-based path planning, and executes smooth trajectories on a 6-DoF collaborative manipulator. This perception-cognition-action pipeline integrates computer
vision, neural artistic abstraction, motion optimization, and
robot control. User ratings on a 5-point scale were high for sketch quality 4.33, perceived execution 4.53, and user experience 4.65, indicating recognizable, appealing, and engaging robotic portraits.
Read the full abstract ▾
All authors Miguel Altamirano Cabrera, Aleksey Fedoseev, Iana Zhura, Dzmitry Tsetserukou
Robotics
RIT*: Riemannian Informed Trees for Cost-Adaptive Optimal Motion Planning
Muhayy Ud Din, Ahmed Nadar, Jan Rosell et al. (+1 more)
We present Riemannian Informed Trees (RIT*), a planning framework that replaces Euclidean primitives in batch-informed search with their Riemannian counterparts. RIT* constructs a tighter, cost-consistent informed set, performs a nearest-neighbour search under an anisotropic distance metric, and evaluates edge costs efficiently via a cascading scheme. We further introduce a Collision-Adaptive Metric Refinement (CARM), which learns an obstacle-proximity cost field online from collision feedback, reducing the reliance on prior metric design in practical settings. Experiments across environments from 2-D to 14-D show that RIT* is competitive in low-dimensional and spatially constant-metric settings and produces substantially lower-cost solutions when the metric varies spatially in high-dimensional configuration spaces. Performance gains scale with anisotropy and dimension, reaching up to 13.0% improvement in median initial cost over BIT* in the 3-D anisotropic
benchmark, up to 9.0% in median final cost over BIT* in 6-DOF manipulation, and 24.8-63.5% in a 14-DOF bimanual planning problem, where Euclidean-informed baselines degrade.
Videos and code can be found here: https://muhayyuddin.github.io/ritstar/
Read the full abstract ▾
All authors Muhayy Ud Din, Ahmed Nadar, Jan Rosell, Irfan Hussain
Robotics
LooperMuscle: Fast and Stable Learning of Humanoid Whole-Body Tracking via Structured Mixture-of-Experts
Boyi Liu, Qijin Li, Tianqi Yu et al. (+2 more)
FastSAC-style methods significantly reduce
humanoid motion training time but often suffer from notable performance degradation compared with PPO in whole-body tracking tasks. We target this speed-performance gap by introducing LooperMuscle, a composed expert policy learning framework that restores tracking quality while preserving high training efficiency. LooperMuscle combines a semantically structured mixture-of-experts actor, an expert-aware distributional critic, and contribution-routed replay with deferred curriculum scheduling. These three components form a closed training loop in which expert contributions guide data routing, routed data shape value learning, and value gradients in turn refine expert specialization. Empirically, our approach substantially outperforms vanilla FastSAC in motion tracking accuracy while requiring far less wall-clock time than PPO: where FastSAC trains in about 15 minutes but underperforms, and PPO achieves stronger results but requires about 6 hours, LooperMuscle recovers a substantial fraction of the remaining gap to PPO in roughly 45 minutes of simulation training, delivering practical efficiency for rapid policy iteration. The code will be released to benefit the research community at https://loopermuscle.github.io/.
Read the full abstract ▾
All authors Boyi Liu, Qijin Li, Tianqi Yu, Qinrui Yan, Xingxing Zuo
Robotics
DynamicWAM: Dual-Path Motion Conditioning for World-Action Models in Dynamic Manipulation
Yunfan Lou, Hewen Gao, Xiyu Zhu et al. (+6 more)
Dynamic manipulation requires
robots to infer target motion and respond promptly, yet existing World-Action Models (WAMs) typically condition only on the current frame and execute large backbones synchronously, limiting motion awareness and responsive control in dynamic scenes. We propose DynamicWAM, a compact WAM for dynamic object manipulation with dual-path motion conditioning. DynamicWAM introduces history-flow conditioning, encoding temporally aligned optical-flow frames alongside the current observation through a frozen pretrained
video VAE to preserve spatial motion structure, while injecting kinematic descriptors of displacement, duration, velocity, and acceleration into the action expert to provide motion magnitude and timing. The two complementary paths are fused through joint world-action
attention. A distilled compact backbone and real-time chunking (RTC)-based asynchronous execution further enable responsive control. On DOMINO, DynamicWAM achieves a 38.2% success rate and a 53.2 manipulation score, outperforming all evaluated baselines. Across 12 real-world tasks spanning linear, circular, and compound target motion, it achieves a 46.7% average success rate, exceeding the strongest baseline by 22.9 percentage points.
Read the full abstract ▾
All authors Yunfan Lou, Hewen Gao, Xiyu Zhu, Zhuoran Qiao, Xuan Han, Yifan Yang, Yifan Ye, Boxian Yao, Zhibo Pang
Robotics
StochSIPP: Safe Interval Path Planning in Stochastic Dynamic Environments
Ajith Kemisetti, Shahaf S. Shperberg and Yoonchang Sung
Safe
navigation under uncertain time-dependent blockage requires anticipating observations before committing to motion. We present StochSIPP, an exact contingent planner for temporal roadmaps with uncertain edge and vertex statuses revealed locally during execution. StochSIPP uses SIPP to generate certified-safe macro-actions that terminate at the next observation or the goal, and bounded AND/OR search over a cached action–observation graph to select actions for every reachable observation outcome. Optimistic and robust SIPP relaxations provide admissible lower and upper bounds for bounded AND/OR search. When every interval declared deterministically safe is truly safe, sensing is exact, and execution follows the planned timing, the resulting policy is provably collision-free. With correct independent probabilities and complete action and outcome generation, it minimizes expected arrival time within the roadmap and horizon. Experiments on controlled roadmap instances show that StochSIPP preserves the observed success of safe fixed-path baselines while reducing arrival time, and solves gated scenarios in which conservative fixed-path planners return no plan. A scalability study further reveals rapid growth as the number of simultaneously observed uncertain statuses increases.
Read the full abstract ▾
Robotics
SIPTraj: Map-Free End-to-End Trajectory Prediction via Physics-Guided Scene Interaction
Feifei Liu, Zejun Wei, Haozhe Wang et al. (+6 more)
Trajectory prediction of surrounding
agents is a prerequisite for safe planning and decision making in autonomous driving. Without high-definition (HD) maps, sensor-derived bird's-eye-view (BEV) features provide no explicit lane
topology or drivable-area priors, making it inherently difficult to ground each
agent in its surrounding scene context. Moreover, physical feasibility remains difficult to capture through data-driven learning alone, as kinematic constraints on
agent motion cannot be explicitly encoded without structured supervision. Existing map-free predictors extract scene context in an
agent-agnostic manner through a single fusion step and treat physical constraints only as output-level penalties, leaving both challenges unaddressed. We propose SIPTraj, a map-free trajectory prediction framework that jointly addresses scene grounding and physical feasibility. SIPTraj introduces a Hierarchical
Agent-Scene Encoder (HASE) progressively grounding each
agent in
agent-guided scene evidence and refining inter-
agent relations within the scene-grounded space. To tackle physical infeasibility in predicted trajectories, we develop a Physics-Guided Iterative Decoder (PGID). It conditions decoding on instantaneous kinematic states, propagating physical supervision into internal representations rather than output trajectories alone. Extensive experiments on nuScenes and Argoverse 2 Sensor show that SIPTraj surpasses prior map-free predictors and strong map-based baselines without any HD map at inference. Our code will be released as open-source.
Read the full abstract ▾
All authors Feifei Liu, Zejun Wei, Haozhe Wang, Yazhi Ye, Yuying Zhang, Jintao Cheng, Chi Man Vong, Xieyuanli Chen, Xiaoyu Tang
Robotics
ORCESTRA: VLM-driven Visual Robot programming in Mixed Reality
Ivan Snegirev, Elizaveta Semenyakina, Mikhail Konenkov et al. (+3 more)
ORCESTRA is a mixed-reality system for programming
robot digital twins through no-code waypoint teaching and
language-guided control. In a passthrough mixed-reality workspace, users place
robot twins on real surfaces, teach trajectories, save
robot-relative episodes, or issue spoken/typed commands that a
vision-
language model converts into structured digital-twin plans. Both interaction modes share a backend for metric grounding, embodiment-aware validation, preview, confirmation, and digital-twin execution. The system supports heterogeneous
robot embodiments, including fixed-base manipulators, a mobile base, and a
humanoid robot, demonstrating MR validation as a safety layer for
language-guided
robot programming before physical deployment.
Read the full abstract ▾
All authors Ivan Snegirev, Elizaveta Semenyakina, Mikhail Konenkov, Artem Lykov, Miguel Altamirano Cabrera, Dzmitry Tsetserukou
Robotics
When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems
Neha Nagaraja, Amisha Bagari and Hayretdin Bahsi
Large
language models are increasingly integrated into autonomous robotic systems for task planning and control, but this integration exposes them to
prompt injection attacks that can lead to unsafe decisions and physical harm. Multi-
agent settings increase the risks through cross-
agent contamination and broader attack surfaces. In this paper, we evaluate
prompt injection attacks against an LLM-based multi-
agent robotic system, considering both direct injections into task instructions and indirect injections through perception modules. In our experiments across varying attack-goal complexities and injection strategies in both single-
agent and multi-
agent settings, we show that
prompt injection can induce
adversarial actions while reducing task completion. We find that attacks can propagate from one
agent to others through shared
prompt structures, with impacts varying depending on
prompt composition and the targeted
agent. We further analyze how architectural changes affect LLM queries and, consequently, the attack success. To the best of our knowledge, this is the first study that systematically investigates
prompt injection attacks in a multi-
agent LLM-based robotic system.
Read the full abstract ▾
Robotics
Push-Wiper: Toward General-Purpose Robotic Cleaning across Varied Stains and Surfaces with Segmented Pushing Trajectories
Renhao Lu, Mingxin Wang, Chenyang Cao et al. (+5 more)
Viscous stains, characterized by high viscosity and complex rheological properties, remain a major challenge for robotic surface cleaning. Conventional wiping often spreads the stain, while scrubbing provides stronger friction but risks damaging the surface. In this paper, we propose Push-Wiper, a framework that reformulates viscous stain cleaning as an aggregation problem. Push-Wiper employs a sponge to progressively gather stains through segmented pushing trajectories, followed by a post-processing phase that detaches the aggregated material and enables sponge self-cleaning. We adopt a stepwise strategy for stain gathering and leverage
Diffusion Policy to generate adaptive pushing action sequences. These sequences are executed through our Arbitrary Surface Pose Interpolator (ASPI) and a hybrid force-position controller, allowing the method to generalize to stains with diverse spatial distributions. Push-Wiper achieves a cleaning score (CS), defined as the percentage of stain area removed, up to 130% higher than baseline methods. Without additional training, Push-Wiper also transfers in a zero-shot manner to solid residues,
liquid spills, unseen viscous stains, and curved surfaces with varying geometries. Our experiments demonstrate the cleaning effectiveness of Push-Wiper and its strong generalization ability. The project website is available at https://push-wiper.github.io/.
Read the full abstract ▾
All authors Renhao Lu, Mingxin Wang, Chenyang Cao, Yang Yang, Guoping Pan, Kangkang Dong, Yi Cheng, Houde Liu
Robotics
SelfWAM: A Self-Grounded Unified World Action Model for Fast Robot Control
Bikang Pan, Fan Liu, Haotao Lu et al. (+2 more)
World Action Models (WAMs) improve
robot policy learning by jointly modeling actions and future observations. However, conditioning future prediction only on the task
prompt and observation context risks capturing generic task progression rather than the action-specific consequences of the executed action. We introduce SelfWAM, a unified self-grounded WAM built on a modality-specialized Mixture-of-
Transformers (MoT) architecture that jointly predicts actions, action-conditioned future RGB frames, and
robot self-masks, thereby grounding future prediction in the
robot's visible body and its action-induced motion. During joint training, SelfWAM allows future visual queries to attend to a clean copy of the demonstrated action, turning the
video branch into an action-specific consequence model while leaving the fast action-only inference path unchanged. To focus
video learning on action-relevant visual changes, we use
prompt-specific objectives for future
robot self-mask prediction, which removes appearance details and provides a target whose temporal
evolution is tightly coupled with the conditioning action. Together, clean-action conditioning and future self-mask supervision make future predictions more directly reflect how the executed action changes the
robot's visible motion and the surrounding scene. Experiments on RoboTwin 2.0 and real-world manipulation tasks show that SelfWAM produces more action-sensitive futures and preserves fast policy inference, while improving policy performance.
Read the full abstract ▾
All authors Bikang Pan, Fan Liu, Haotao Lu, Jingya Wang, Ye Shi
Robotics
OmniAI: A Surface-Adaptive Aerial Projection Interface for Human–Drone Interaction
Nikita Kuzmin, Yuhua Jin, Georgii Demianchuk et al. (+6 more)
Drones in human environments often lack spatially grounded in- terfaces for situated communication. We present OmniAI, an em- bodied aerial
agent that supports surface-adaptive interaction by switching projection between an onboard screen and nearby en- vironmental surfaces. A servo-actuated MEMS
laser projector renders text-and-image responses from a web-augmented LLM pipeline. Projection surfaces are detected online using RGB-D sensing and RANSAC plane fitting, without pre-mapped geometry. OmniAI provides functionally equivalent voice and gesture con- trol for both drone motion and projected content. By combining
speech, mid-air gestures, adaptive projection, and aerial mobility, OmniAI demonstrates a mobile spatial AR interface for context- aware human-drone interaction.
Read the full abstract ▾
All authors Nikita Kuzmin, Yuhua Jin, Georgii Demianchuk, Mariya Lezina, Fawad Mehboob, Ivan Valuev, Nikolai Lutsenko, Miguel Altamirano Cabrera, Dzmitry Tsetserukou
Robotics
Staged Multi-Agent Training (SMAT) for Hip Exoskeletons: Metabolic and Biomechanical Validation of a Simulation-Trained Co-Adaptive Controller
Yifei Yuan, Jakob Wolf, Ghaith Androwis et al. (+1 more)
Learning-based controllers can deliver exoskeleton assistance after training entirely in physics-based simulation, yet few controllers that address human-device co-adaptation have been validated on real users by whole-body metabolic measurement, the standard
benchmark for assistive walking. Co-adaptation is challenging: as the device alters joint dynamics, the wearer reorganizes neuromuscular coordination, producing a non-stationary learning problem. Staged Multi-
Agent Training (SMAT), a four-stage curriculum that progressively trains a musculoskeletal human actor and a bilateral hip exoskeleton actor, was introduced and shown to reduce simulated hip-muscle activation and provide positive assistance on hardware. This article provides the first physiological validation of SMAT. The policy was deployed on a hip exoskeleton and tested with eight healthy adults, with metabolic cost measured by indirect calorimetry across no-exoskeleton, passive, and active conditions. Active assistance lowered net metabolic rate by 19.7% relative to the passive device (p < 0.001). Biomechanical analysis confirmed predominantly positive hip mechanical power across all subjects (positive-power ratio 0.98), and the policy generalized across walking speeds and terrains. Together, these results show that a single simulation-trained SMAT policy, deployed without subject-specific retraining, delivers a significant metabolic benefit on real users while remaining robust beyond the conditions it was trained on.
Read the full abstract ▾
All authors Yifei Yuan, Jakob Wolf, Ghaith Androwis, Xianlian Zhou
Robotics
Assistant Placement Aria: A Benchmark for Egocentric Placement Assistance
Amir Belder, Gonçalo Dias Pais, Refael Vivanti et al. (+5 more)
Human assistance in robotics spans around several tasks such as
navigation, object manipulation, and placement, where a key challenge is selecting target destinations that align with human intentions or preferences. We focus on this challenge in the context of Virtual Placement (VP), the task of identifying all plausible target locations given scene context and human-centric constraints. This differs from traditional placement tasks that typically focus on a single, predefined target location. The VP problem is complex, as it requires both global and local
reasoning about the scene's geometry, semantics, and plausibility. To address this gap, we introduce Assistant Placement Aria, the first
benchmark to explore diverse aspects of VP, including global, local, and human-centric constraints. It contains both synthetic and real indoor scenes annotated for three tasks: (i)~2D Panel Placement, (ii)~Sitting Suggestion, and (iii)~TV Placement. Each scene includes 2D images, a 3D point cloud, and a textual description of the objects within the scene. By contributing this
benchmark, we aim to encourage further research in this underexplored and challenging field that is critically dependent on relevant data. We also evaluate several foundation models for object detection and
segmentation on our
benchmark.
Read the full abstract ▾
All authors Amir Belder, Gonçalo Dias Pais, Refael Vivanti, Omri Carmi, Daniel DeTone, Oren Shrout, Ido Gattegno, Ayellet Tal
Robotics
FlowPilot: Real-Time World-Action Modeling for Agile UAV Navigation
Runqing Wang, Ding Yu, Pengyuan Min et al. (+6 more)
We present FlowPilot, a compact world-action model for real-time onboard
UAV navigation from depth. Unlike map-then-optimize pipelines that require local reconstruction or end-to-end policies that lack explicit scene prediction, FlowPilot jointly denoises future depth observations and executable trajectories with flow matching. A dual-stream mixture-of-
transformers couples
video and action experts through shared
attention, allowing future-scene prediction and trajectory generation to inform each other. At deployment, the model runs action-centrically and outputs only a trajectory. To ensure trackability, actions are parameterized as degree-7 Bernstein
polynomials: the current state constrains the initial control points, and the network predicts five free control points, yielding C^2-continuous references with closed-form velocity, acceleration and jerk. FlowPilot is trained on a three-level depth pyramid spanning high-throughput simulation, photorealistic simulation, and real onboard data. In closed-loop simulation, it outperforms learning- and optimization-based baselines under increasing clutter and commanded speeds up to 8m/s. On a physical quadrotor, the full perception-to-action pipeline runs in under 18ms on a Jetson Orin NX and reaches 5.5m/s in cluttered indoor and forest environments using only onboard sensing and computation.
Read the full abstract ▾
All authors Runqing Wang, Ding Yu, Pengyuan Min, Xinhong Zhang, Wei Xiao, Yu Hu, Jie Chen, Fu Zhang, Gang Wang
Robotics · Cross-listed from cs.HC
bFaaaP: An Inclusive, Head-Angle Piano-Pedal Interaction that Quantitatively Reproduces a Pianist's Intended Pedalling – Foot-Free, for Acoustic and Electronic Pianos
Tomoyuki Shishido, Masahiro Ootaki and Hiroyuki Narusawa
Expressive piano performance depends on the sustain (damper) pedal, operated by foot, excluding players who cannot readily use their feet: wheelchair users and others with lower-limb impairments, small children, and some elderly or disabled players. We present bFaaaP (barrier-Free assist as a Pedal), an inclusive, foot-free interaction that operates the pedal from the angle of the player's head: a smartphone tracks head pose with on-device augmented-reality (AR) face tracking and streams a compact command over Bluetooth Low Energy (BLE) to a pedal device. Supported by patent examination, our central claim is not the head-to-pedal architecture (anticipated by prior art) but a quantitative, user-tunable control law – the patentable "key" to a natural, expressive result: the player presets a small angular dead-zone (offset 3-10 degrees) and a multiplier (10-50), which together fix a secondary, pre-adjustable response speed that reproduces the pianist's intended pedalling. An engineering trick decouples the fast AR sampling from the slower BLE rate. Two co-equal realizations share one controller: a non-destructive robotic actuator for acoustic pianos (Pro), anchored by a pneumatic "airback" (our coined term for an inflatable air-braced anchor) that absorbs the reaction force without modifying the instrument; and an electronic sustain switch for digital pianos (Switch). In a human-subject Auxiliary Pedal Effect Evaluation (APEE) with 15 participants, bFaaaP significantly increased sustained-tone energy (p<0.01) and was statistically indistinguishable from a player's own foot (p>0.05), with no significant difference across classes; one participant with a leg disability and a tracheostomy performed successfully. With nothing worn on the face and fast setup, bFaaaP has run in formal public concerts (2018-2025). We release the full hardware and software as open source.
Read the full abstract ▾
Robotics
Learning-Based Motion Planning for Dynamic Environments: From Foundational Algorithms to Emerging Paradigms
Zongyuan Shen, Shalabh Gupta, Shancheng Zhao et al. (+7 more)
Motion planning in dynamic environments is a fundamental problem in robotics, aiming to generate safe and efficient paths, trajectories, or control actions in the presence of moving obstacles, uncertain predictions, and multi-
agent interactions. It has broad applications in autonomous driving, service robotics, warehouse logistics, human-
robot collaboration, crowd
navigation, and multi-
robot systems. This survey reviews representative works published primarily between 2015 and 2025, with a particular focus on how recent learning-based advances extend, complement, or interact with classical planning foundations. We first revisit classical planning methods as algorithmic foundations and reference frameworks for learning-based extensions. We then propose a role-of-learning taxonomy that categorizes existing methods according to how learning participates in the planning pipeline, including direct policy learning, learning-augmented classical planning, hybrid planning, and training enhancement methods. For each category, we summarize the main problem settings, representative
algorithms, key ideas, integration mechanisms, strengths, and limitations. We further analyze how observation representations, prediction uncertainty, interaction modeling, planner integration, safety constraints, and training strategies shape learning-based motion planning in dynamic environments. Finally, we discuss open challenges and future directions, including sim-to-real gap, safe and certifiable planning, dense crowd
navigation, perception-planning coupling, and
embodied AI.
Read the full abstract ▾
All authors Zongyuan Shen, Shalabh Gupta, Shancheng Zhao, Dehua Zhou, Gao Wang, Rui Cheng, Yaming Ou, Zhongqiang Ren, Yikui Zhai, C. L. Philip Chen
Robotics
From Failures to Supervision: DynamicEnvPlan for Robust Long-Horizon Embodied Planning
Hao Yuan, Yuxin Wang, Lei Ji et al. (+1 more)
Physical-world interaction is inherently dynamic, as environments can evolve during execution, requiring
agents to adapt their plans under non-stationary conditions. We study this challenge through long-horizon
embodied planning under environment deviations and execution uncertainty. Existing
embodied-task
benchmarks can expose such failures, but these failures are usually treated as evaluation outcomes instead of learnable signals for training
agents to recover. In this work, we introduce DynamicEnvPlan, a closed-loop framework for high-level planning in dynamic environments. It extends
embodied task execution with
humanoid agents, high-level primitive skills, structured semantic
memory, and controllable perturbations. Our data synthesis design consists of planning, perturbation, and guarded correction modules that turn dynamic execution states into recovery-oriented traces. The resulting traces are used for staged supervised fine-tuning, enabling the planner to learn from both nominal execution and perturbed recovery trajectories. Using 104 task-scene combinations spanning i.i.d., compositional generalization, and out-of-distribution settings for fine-tuning and evaluation, DynamicEnvPlan boosts success rate from 33.3% for the base planner to 76.2%, while improving across all seven evaluation metrics critical to physical-world interaction, including safety and affordance compliance.
Read the full abstract ▾
All authors Hao Yuan, Yuxin Wang, Lei Ji, Zhiwei Yu
Robotics
Grasp Execution Without a Planner: Configuration-Space Grasp Distance Fields with Certified Safety & Guaranteed Quality
Clinton Enwerem, John S. Baras and Calin Belta
Mainstream plan-then-track approaches to multifingered grasp execution entail selecting a grasp, planning a collision-free trajectory, and tracking the resulting trajectory via a feedback controller. Pose-estimation error during execution or scene motion can invalidate this open-loop commitment and trigger replanning. We thus present Grasp Distance Fields (GDFs), smooth softmin distances to finite sets of arm-hand grasp configurations. Using their negative gradients as feedback, we jointly select and execute grasps without planning a trajectory. A CBF-CLF quadratic program (QP) enforces self-collision, workspace, object, and obstacle-clearance constraints, while its CLF slack quantifies obstruction of task progress. We bound the softmin approximation error by $\log N/\rho$ and prove forward invariance of the filtered safe set. To handle changes in contact
topology, we combine a hysteretic contact-mode transition with a wrench-quality CBF that limits degradation of the realized force-closure margin relative to hold onset. Using our method, a fixed-base manipulator and a Unitree G1 equipped with the same underactuated hand grasp and lift 46 of 50 test objects amid clutter and moving obstacles. The realized grasps also retain a median 94% of their synthesized quality margin, and each QP solve requires 0.09 ms within a 20 ms control interval. Project page: www.clintonenwerem.com/gdf.
Read the full abstract ▾
Robotics
Latency-Tolerant Cloud-Edge Collaborative Vision-Language-Action Models via Emergent Representational Specialization
Daojie Peng, Fulong Ma, Bingtao Wang et al. (+2 more)
Deploying billion-parameter
Vision-
Language-Action (VLA) policies on mobile
robots creates a systems conflict: semantic
reasoning benefits from cloud GPUs, whereas closed-loop control must respond locally despite network delay and jitter. Existing hierarchical and asynchronous policies improve throughput, but their slow-path representations can still arrive stale or require explicit scheduling and delay cues. We introduce CloudEdgeVLA, a cloud-edge policy that treats temporal misalignment as a representation-learning problem. A cloud VLA encodes delayed observations into slowly varying task features, while a lightweight edge head combines the latest available cloud feature with current local
vision. During training, current and randomly delayed frames are paired with the same current action target in fresh and stale paths. This objective encourages the cloud representation to preserve task-level information while the edge path supplies state-sensitive corrections, driving emergent specialization. Across four LIBERO suites, CloudEdgeVLA retains 63.8-78.0% success with a 40-step uniform-delay window, whereas VLASH reaches at most 6.4% and the evaluated single-path baselines at most 3.0%. By removing blocking synchronization from the control loop, the design offers a practical route to scalable VLA deployment in which cloud models can grow while edge computation remains lightweight and responsive.
Read the full abstract ▾
All authors Daojie Peng, Fulong Ma, Bingtao Wang, Sheng Wang, Jun Ma
Robotics
DexMani: Human-Derived Manipulability Guidance for Dexterous Rotation
Xiaoyang Chen, Shengcheng Luo, Haoran Guo et al. (+4 more)
Dexterous object rotation is a sequential contact problem: each support, release, and re-contact decision must both produce the desired object motion, and prepare the hand configuration for continued rotation. Existing reinforcement learning methods discover such movement patterns through trial and error on specific robotic hand embodiments, without explicitly accounting for how each contact transition affects the hand's ability to sustain object rotation in subsequent steps. We introduce DexMani, a framework that transfers human demonstrations as contact-conditioned manipulability
evolution. This prior captures how successful human contact transitions reshape the object-rotation directions available to the hand. DexMani then learns this manipulability
evolution and uses it to guide downstream reinforcement learning, enabling rotation skills to be acquired across
robot embodiments with distinct kinematics and active-contact configurations. Across the Shadow Hand, Allegro Hand, and XHand, DexMani achieves the highest success rates in every evaluated setting for both seen and unseen objects. DexMani reaches an average success rate of 57.5% on LEAP Hand, outperforming other baselines and producing smoother rotatory motions. Project site: https://dexmani.github.io
Read the full abstract ▾
All authors Xiaoyang Chen, Shengcheng Luo, Haoran Guo, Jiaming Jiang, Wanlin Li, Ziyuan Jiao, Chenxi Xiao
Robotics
Disentangling Visuo-Tactile Foresight: Oracle-Guided Interface Discovery for World Action Models
Zihang Yao, Chaoyue Ding and Yingying Yu
Contact-rich manipulation remains challenging because successful control depends on physical interaction cues that are often weakly observable from
vision alone. Recent tactile world action models jointly model future visual observations and tactile signals to guide action generation, but how such futures should be structured for effective use by the action expert remains underexplored. Directly studying this question with learned world action models is difficult because end-to-end behavior entangles physically invalid visual futures, unreliable predictions, inaccurate or cross-modally inconsistent tactile forecasts, and an unreadable future-to-action interface. To make this interface independently studyable, we introduce Oracle Visuo-Tactile Foresight (OVTF), a controlled framework that supplies paired RGB and tactile futures from successful trajectories verified in simulation. By fixing the future provider, OVTF isolates the interface and asks a cleaner question: if the future is successful and physically executable, what representation allows the action expert to absorb its benefit? Within OVTF, we propose Asymmetric Phase-Local Future
Memory (AFM), in which visual
memory reads future
vision, each tactile
memory jointly attends to its own tactile stream and phase-aligned future
vision, and cross-tactile access is blocked. We compare AFM with Modality-Isolated Future
Memory (IFM), which removes visual-to-tactile access and processes each future modality independently. Across seven tasks on the UniVTAC simulation
benchmark, AFM achieves 32.0% average success, compared with 23.7% for IFM and 14.9% for UniVTAC-ACT. This controlled comparison shows that selective phase-aligned visual-tactile routing provides a more actionable future-to-action bridge than complete modality isolation.
Read the full abstract ▾
Robotics
SSTG-Nav: Metric-Grounded Spatial-Semantic Topological Graphs for Reusable Object Navigation
Daojie Peng, Bingtao Wang and Jun Ma
Service
robots operating for months in the same homes, offices, and facilities should become more reliable with experience instead of searching familiar space from scratch for every request. Yet ObjectNav is predominantly formulated as one-shot exploration, leaving a central deployment challenge unresolved: recognizing an object does not identify a reachable place to stop, and one confident map error can terminate the task. We introduce SSTG-Nav, a reusable metric-semantic
memory that turns a one-time survey into actionable object goals, consolidates evidence across viewpoints, and retains spatially distinct recovery standoffs. On 1,000 HM3D-v2 episodes across 36 scenes, our goal-independent
topology achieves a 99.4%
geometric success ceiling. Holding semantic responses fixed, metric grounding raises SR/SPL from 0.835/0.560 to 0.920/0.603, and source-aware fusion reaches 0.926/0.586. Fusion-aware Top-3 recovery raises Success@1/2/3 to 0.928/0.965/0.975 and reaches 0.601 SPL@3. Model, field-of-view, density, and corruption controls identify where these gains originate, and a ROS2/Nav2 realization demonstrates the complete reusable query-to-execution pipeline. Together, the results establish pre-exploration as a powerful practical regime for dependable, repeated semantic
navigation.
Read the full abstract ▾
Robotics
A Change of Frame Makes the Capture Point Proprioceptive: Distillation-Free Humanoid Single-Leg Balance
Yikai Zhou, Xingyun Wang, Jieming Cui et al. (+4 more)
Unified
humanoid policies track agile whole-body motion, yet few can hold a clean single-leg stance. On the single-leg balance
benchmark introduced here, eight released state-of-the-art general policies hold a clean stance on none of 90 held-out motions; they stay upright only by hopping or re-planting a foot, recovering from imbalance rather than preventing it. Prevention needs the capture point, the center of mass (CoM) extrapolated by its velocity. That velocity contains the base linear velocity, which no on-board sensor measures, so the capture point has been confined to rewards and privileged critics and has reached hardware only through teacher–student
distillation. A change of frame removes the obstacle: expressed relative to the support foot, the base velocity cancels identically, leaving a capture-point state reconstructible from joint encoders and an inertial measurement unit alone. We place this support-relative dynamic-CoM observation directly in the deployed actor and pair it with a reward library translated term by term from human postural control. Trained with asymmetric FastSAC and no
distillation, the resulting policy, DDC, holds clean single-leg balance on 89 of 90 held-out motions across nine pose classes and runs directly on a Unitree G1; removing the observation alone costs 43 points, and 53 under deployment noise. We release the policy, the data, and a method-agnostic MuJoCo
benchmark for
humanoid single-leg balance, which scores released policies on the same held-out motions. Together these turn single-leg balance from a per-task demonstration into a capability the field can measure and build into general policies.
Read the full abstract ▾
All authors Yikai Zhou, Xingyun Wang, Jieming Cui, Bozhou Chen, Yikai Fan, Yixin Zhu, Wenxin Li
Robotics
From Digital to Physical Reservoir Computing: Co-Optimizing Soft Robotic Reservoirs via Dynamics Matching
Nicola Visentin, Maximilian Stölzle, Mariano Ramírez Montero et al. (+3 more)
Soft robotic substrates are promising for Physical Reservoir Computing (PRC) because their compliant nonlinear dynamics can provide temporal
memory, high-dimensional state transformations, and efficient inference. However, physical reservoirs are often adopted as-is rather than pretrained or co-optimized, potentially limiting soft robotic PRC performance relative to digital reservoirs. We investigate whether a physical reservoir can instead be pretrained against high-performing digital reference dynamics. Our formulation jointly optimizes physical parameters, a diffeomorphic physical-reference state map, and feedforward-feedback control using a differentiable physical model and an acceleration-level equation-error objective that avoids temporal integration. As a proof of concept, we instantiate the formulation with simulated soft
robots, a Random Oscillators Network (RON) reference, and parallel multi-start gradient descent. We evaluate the optimized reservoirs on classification (sMNIST and ADIAC) and
forecasting (Mackey-Glass and Lorenz96) tasks across four reservoir dimensions. Compared with unoptimized soft
robot reservoirs, the optimized reservoirs achieve a mean relative improvement of 33.7% across all tasks and
datasets, while remaining close to the digital reference. These results demonstrate the feasibility of dynamics-level co-optimization for the simulated soft robotic reservoirs considered here.
Read the full abstract ▾
All authors Nicola Visentin, Maximilian Stölzle, Mariano Ramírez Montero, Francesco Braghin, Daniela Rus, Cosimo Della Santina
Robotics
Learning to Predict Contact Force Distributions from Vision Leveraging Object Geometry Priors
Ryo Hanai, Yukiyasu Domaea, Ixchel G. Ramirez-Alpizar et al. (+3 more)
Based on
vision and prior experience, humans can make rough physical predictions and adjust their manipulation strategies. This paper aims to endow
robots with a similar ability. To collect paired data of
vision and forces, we use a rigid-body simulator commonly adopted in robotics. However, unlike simulators that output noisy point forces, humans are able to make consistent predictions even in unfamiliar situations. Based on this observation, we hypothesize that predicting smooth force distributions rather than raw point forces can improve both force prediction itself and downstream task performance. To validate this hypothesis, we construct a model that predicts three-dimensional force distributions from a single RGB image of piled daily objects. The target distribution is generated by applying statistical smoothing to point forces obtained from the simulator. Moreover, by incorporating object geometry into the smoothing process, we aim to account for variations in contact states and achieve more consistent
vision-based predictions. We conduct extensive evaluations in both simulation and real environments. Results show that our approach improves prediction accuracy, enhances downstream task performance through smoothing, and further benefits from geometry-guided smoothing. Remarkably, the trained model generalizes effectively to real-world scenes despite being trained solely in simulation.
Read the full abstract ▾
All authors Ryo Hanai, Yukiyasu Domaea, Ixchel G. Ramirez-Alpizar, Abdullah Mustafa, Floris Erich, Tetsuya Ogata
Robotics · Cross-listed from eess.SP
LOCUS-DT: Localization via Observation-Conditioned Uncertainty Scoring with Digital Twins
Haozhe Lei, Roberto Bomfin, Marwa Chafii et al. (+1 more)
Accurate indoor localization is essential for emerging applications in robotic
navigation and search and rescue. While classical methods typically focus on single-point estimates, complex indoor environments with heavy blockage and multipath propagation often lead to
multimodal likelihood surfaces where a single estimate is insufficient. This paper proposes LOCUS-DT (Localization via Observation-Conditioned Uncertainty Scoring with Digital Twins), a framework that treats snapshot localization as posterior inference over the transmitter location. By leveraging a ray-tracing-based digital twin (DT) of the known environment, LOCUS-DT generates synthetic multipath profiles for candidate locations and compares them against the measured channel profile. Central to our approach is a novel learned scoring function designed to compare a fixed number of dominant specular paths, providing robustness against errors in both the DT environment model and the physical channel estimation. Importantly, LOCUS-DT is trained over an ensemble of environments to ensure generalization to unseen layouts. We evaluate the system using a Sionna-based ray-tracing backend, demonstrating that LOCUS-DT captures the sharp,
multimodal posterior structures inherent in indoor settings more accurately than standard Gaussian or Gaussian-mixture
benchmarks.
Read the full abstract ▾
All authors Haozhe Lei, Roberto Bomfin, Marwa Chafii, Sundeep Rangan
Robotics
Embodied Passive Aeroacoustic Perception Enables Relative Sensing and Pursuit Between Aerial Robots
Yanbaihui Liu, Ravi Prakash, Li-Yu Lo et al. (+2 more)
Aerial
robots generate structured aeroacoustic fields during flight, yet these signals have been underexplored as a source of onboard relative perception, particularly under the strong ego-acoustic interference generated during simultaneous flight in various outdoor conditions. We introduce
embodied passive aeroacoustic perception, a sensing paradigm in which an aerial
robot infers actionable relative-state information from the naturally generated sound of flight while operating within its own evolving aeroacoustic field. We present SonicFly, a passive aeroacoustic perception framework that enables one unmanned aerial vehicle to estimate and follow another using only the leader's intrinsic flight sound, without active acoustic signaling, inter-
robot communication, GPS sharing, or external sensing infrastructure. The system uses a lightweight four-microphone array, rotorcraft-informed acoustic representations, a neural bearing-range
estimator, and confidence-gated filtering for closed-loop flight. Through acoustic characterization, onboard localization, and outdoor pursuit experiments, we show that multirotor aeroacoustic signals contain sufficient information to support relative perception despite strong ego-acoustic interference, environmental variability, and changing flight geometry. During acoustic-only pursuit, SonicFly achieved a mean distance-maintenance error of 1.34 m across diverse outdoor trajectories and operating conditions. Analysis of the acoustic channel reveals design principles governing
embodied passive aeroacoustic perception, including the roles of harmonic structure, spectral separability, and spatial acoustic cues in determining observability. Our results establish the feasibility of
embodied passive aeroacoustic perception for aerial
robots and suggest that naturally generated behavioral signals can serve as information for robotic perception and coordination.
Read the full abstract ▾
All authors Yanbaihui Liu, Ravi Prakash, Li-Yu Lo, Nils Roede, Boyuan Chen