Robotics
Good in Bad (GiB): Sifting Through End-user Demonstrations for Learning a Better Policy
Noushad Sojib, Ola Ghattas, Sajay Arthanat et al. (+1 more)
Imitation learning offers a promising framework for enabling
robots to acquire diverse skills from human users. However, most imitation learning
algorithms assume access to high-quality demonstrations an unrealistic expectation when collecting data from non-expert users, whose demonstrations often contain inadvertent errors. Naively learning from such demonstrations can result in unsafe policy behavior, while discarding entire demonstrations due to occasional mistakes wastes valuable data, especially in low-data settings. In this work, we introduce GiB (Good-in-Bad), an
algorithm that automatically identifies and discards erroneous subtasks within demonstrations while preserving high-quality subtasks. The filtered data can then be used by any policy learning
algorithm to train more robust policies. GiB first trains a self-supervised model to learn latent features and assigns binary weights to label each demonstration as good or bad. It then models the latent feature distribution of high-quality segments and uses the Mahalanobis distance to detect and evaluate poor-quality subtasks. We validate GiB on the Franka
robot in both simulated and real-world multi-step tasks, demonstrating improved policy performance when learning from mixed-quality human demonstrations.
Read the full abstract ▾
All authors Noushad Sojib, Ola Ghattas, Sajay Arthanat, Momotaz Begum
Robotics
VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids
Zichao Hu, Zifan Xu, Dongsik Chang et al. (+6 more)
The ability to push large objects in a goal-directed manner using onboard
egocentric perception is an essential skill for
humanoid robots to perform complex tasks such as material handling in warehouses. To robustly manipulate heavy objects to arbitrary goal configurations, the
robot must cope with unknown object mass and ground friction, noisy onboard perception, and actuation errors; all in a real-time feedback loop. Existing solutions either rely on privileged object-state information without onboard perception or lack robustness to variations in goal configurations and object physical properties. In this work, we present VOFA, a visual goal-conditioned
humanoid loco-manipulation system capable of pushing objects with unknown physical properties to arbitrary goal positions. VOFA consists of a two-level hierarchical architecture with a high-level visuomotor policy and a low-level force-adaptive whole-body controller. The high-level policy processes noisy onboard observations and generates goal-conditioned commands to operate in closed loop across diverse object-goal configurations, while the low-level whole-body controller provides robustness to variations in object physical properties. VOFA is extensively evaluated in both simulation and real-world experiments on the Booster T1
humanoid robot. Our results demonstrate strong performance, achieving over 90% success in simulation and over 80% success in real-world trials. Moreover, VOFA successfully pushes objects weighing up to 17kg, exceeding half of the Booster T1's body weight.
Read the full abstract ▾
All authors Zichao Hu, Zifan Xu, Dongsik Chang, He Yin, Linh Tran, Roberto Martín-Martín, Peter Stone, Jingyu Qiao, Joydeep Biswas
Robotics
Dynamics Distillation for Efficient and Transferable Control Learning
Xunjiang Gu, Kashyap Chitta, Mahsa Golchoubian et al. (+2 more)
Robust control policy learning for autonomous driving requires training environments to be both physically realistic and computationally scalable, properties that existing simulators provide only in isolation. We introduce Sim2Sim2Sim, a framework that bridges high-fidelity vehicle simulation and scalable reinforcement learning by distilling simulator dynamics into a highly parallelizable learned dynamics model. By training control policies purely within this distilled environment and deploying them back into the high-fidelity source simulator, we demonstrate more efficient policy optimization and reliable transfer under challenging dynamics. We further show that predictive accuracy alone does not fully characterize a learned dynamics model's suitability as a reinforcement learning training environment, which should also be assessed by the quality of the policies it enables.
Read the full abstract ▾
All authors Xunjiang Gu, Kashyap Chitta, Mahsa Golchoubian, Vladimir Suplin, Igor Gilitschenski
Robotics
Distributed Algorithm with Emergent Area Partitioning and Base Station's Situation Awareness for Multi-Robot Patrolling
Kazuho Kobayashi, Shohei Kobayashi, Seiya Ueno et al. (+1 more)
Patrolling with multiple
robots offers efficient surveillance to detect and manage undesired situations. This necessitates improved patrol efficiency and operator situation awareness at base stations. Enhanced situation awareness enables operators to predict
robots' behaviors, support recognition and decision-making, and execute emergency interventions. This study presents the Local Reactive and Partition (LR-PT)
algorithm, a novel multi-
robot patrolling approach. In simulations, LR-PT outperformed existing methods by ensuring frequent patrols of all locations of interest and enhancing the situation awareness of the base station.
Robots independently select patrol targets based on locally available information, integrating patrol needs and the urgency of reporting mission progress to the base station into a unified utility function. This locality also contributes to robustness against communication constraints and
robot failures, as demonstrated in this research. The
algorithm further autonomously emerged the area partition, which can avoid falling into local optima and realize the comprehensive patrol over the whole mission area. The simulation results demonstrated the superior performance of LR-PT for multi-
robot patrolling, utilizing the advantages of swarm robotics and addressing real-world operational challenges.
Read the full abstract ▾
All authors Kazuho Kobayashi, Shohei Kobayashi, Seiya Ueno, Takehiro Higuchi
Robotics
Cut-In Gap Acceptance Toward Autonomous vs. Human-Driven Vehicles: Evidence from the Waymo Open Motion Dataset
Abdulaziz Alhuraish, Yuhang Wang and Hao Zhou
Autonomous vehicles (AVs) are widely known to follow conservative, rule-based motion policies that surrounding drivers can learn to anticipate. A direct consequence is that human drivers may accept shorter longitudinal gaps when cutting in front of an AV than when targeting another human-driven vehicle (HDV). We test this hypothesis using the Waymo Open Motion
Dataset (WOMD), which provides 25,906 real-world highway scenarios at 10 hertz. An eight-criterion lane-change detector extracts 706 HDV-to-AV and 3,172 HDV-to-HDV cut-in events from the same traffic environment. The median accepted gap in front of the Waymo AV is 7.58 meters versus 9.57 meters for HDV targets, a 1.99 meter reduction that is statistically significant (p equals 5.76 times 10 to the negative eighth power, d equals negative 0.224) and persists under speed-matched resampling. Cut-in speeds toward the AV are 37 percent higher (51.7 versus 37.7 kilometers per hour, d equals 0.502), and 68.0 percent of AV-targeted cut-ins occur below the 10 meter gap boundary versus 51.8 percent of HDV-targeted events (chi-squared equals 60.5, p is less than 10 to the negative thirteenth power). These results reveal a systematic and safety-relevant asymmetry in human gap-acceptance behavior that warrants AV-specific calibration of both motion-planning safety envelopes and traffic simulation models.
Read the full abstract ▾
Robotics
Action Agent: Agentic Video Generation Meets Flow-Constrained Diffusion
Jeffrin Sam, Nguyen Khang, Yara Mahmoud et al. (+2 more)
We present Action
Agent, a two-stage framework that unifies agentic
navigation video generation with flow-constrained
diffusion control for multi-embodiment
robot navigation. In Stage I, a large
language model (LLM) acts as an orchestration module that selects
video diffusion models, refines
prompts through iterative validation, and accumulates cross-task
memory to synthesize physically plausible first-person
navigation videos from
language and image inputs. This increases
video generation success from 35% (single-shot) to 86% across 50
navigation tasks. In Stage II, we introduce FlowDiT, a Flow-Constrained
Diffusion Transformer that converts optimized goal
videos and
language instructions into continuous velocity commands using action-space denoising
diffusion. FlowDiT integrates DINOv2 visual features, learned optical flow for ego-motion representation, and CLIP
language embeddings for semantic stopping. We pretrain on the RECON outdoor
navigation dataset and fine-tune on 203 Unitree G1
humanoid episodes collected in Isaac Sim to calibrate velocity dynamics. A single 43M-parameter checkpoint achieves 73.2%
navigation success in simulation and 64.7% task completion on a real Unitree G1 in unseen indoor environments under open-loop execution, while operating at 40–47 Hz. We evaluate Action
Agent across three embodiments: a Unitree G1
humanoid (real hardware), a drone, and a wheeled mobile
robot (Isaac Sim), demonstrating that decoupling trajectory imagination from execution yields a scalable and embodiment-aware paradigm for
language-guided
navigation.
Read the full abstract ▾
All authors Jeffrin Sam, Nguyen Khang, Yara Mahmoud, Miguel Altamirano Cabrera, Dzmitry Tsetserukou
Robotics
LLM-Foraging: Large Language Models for Decentralized Swarm Robot Foraging
Peihan Li, Joanna Gutierrez, Fabian Hernandez et al. (+2 more)
Swarm foraging
algorithms, such as the central-place foraging
algorithm (CPFA), typically rely on offline parameter optimization using genetic
algorithms (GA) or reinforcement learning, yielding policies tightly coupled to a specific combination of team size, arena size, and resource distribution. When deployment conditions change, performance degrades, and retraining is computationally expensive. We propose LLM-Foraging, a decentralized swarm controller that augments the CPFA state machine with a large
language model (LLM) tactical decision-maker at three structured decision points, namely post-deposit, central-zone arrival, and search starvation. Each
robot runs its own LLM client and queries it using only locally observable state, while the existing CPFA motion and sensing stack executes the selected action. Because the LLM serves as a general decision policy rather than parameters fitted to a single configuration, the controller is training-free at deployment and transfers across configurations without re-optimization. We evaluate LLM-Foraging in Gazebo with TurtleBot3
robots across 36 configurations spanning team sizes of 4 to 10
robots, arena sizes from 6x6 to 10x10 meters, and three resource distributions (clustered, powerlaw, random). LLM-Foraging collects more resources than the GA-tuned CPFA baseline across the evaluated configurations and is more consistent, a property that the GA's single-configuration tuning does not transfer.
Read the full abstract ▾
All authors Peihan Li, Joanna Gutierrez, Fabian Hernandez, Qi Lu, Lifeng Zhou
Robotics
Decompose and Recompose: Reasoning New Skills from Existing Abilities for Cross-Task Robotic Manipulation
Xitie Zhang, Aming Wu and Yahong Han
Cross-task generalization is a core challenge in open-world robotic manipulation, and the key lies in extracting transferable manipulation knowledge from seen tasks. Recent in-context learning approaches leverage seen task demonstrations to generate actions for unseen tasks without parameter updates. However, existing methods provide only low-level continuous action sequences as context, failing to capture composable skill knowledge and causing models to degenerate into superficial trajectory imitation. We propose Decompose and Recompose, a skill
reasoning framework using atomic skill-action pairs as intermediate representations. Our approach decomposes seen demonstrations into interpretable skill–action
alignments, enabling the model to recompose these skills for unseen tasks through compositional
reasoning. Specifically, we construct a task-adaptive dynamic demonstration library via visual-semantic
retrieval combined with skill sequences from a planning
agent, complemented by a coverage-aware static library to fill missing skill patterns. Together, these yield skill-comprehensive demonstrations that explicitly elicit compositional
reasoning for skill composition and execution ordering. Experiments on the AGNOSTOS
benchmark and real-world environments validate our method's zero-shot cross-task generalization capability.
Read the full abstract ▾
Robotics
High-Speed, Scalable Sensor Readout for Dexterous Robotic Hands via Shift-Register Multiplexing
Jaehoon Kim, Boshi An, Lazaros Christoforidis et al. (+3 more)
Dexterous robotic hands require high-speed
multimodal sensing across many degrees of freedom, yet existing readout architectures often impose trade-offs between sensor count, wiring complexity, and sampling bandwidth. This paper presents a scalable analog sensor readout architecture based on a serial-in parallel-out (SIPO) shift-register principle. The proposed architecture supports versatile integration of heterogeneous analog-output sensors, scalable expansion using only three signal lines between sensor modules, and fast, configurable sampling. We validate the approach on a tendon-driven robotic hand integrating 16 joint sensor modules and one four-channel tactile sensor module, enabling acquisition of 20 sensor channels at a full-scan rate of 1 kHz, with stable operation up to 1.5 kHz. Joint sensor characterization showed a maximum slope absolute percentage error (APE) of 0.446% and sub-degree estimation error, indicating that the proposed readout system preserves sensing performance. For tactile sensing, LSTM-based models enabled real-time force estimation and contact-location classification at 1 kHz. System-level experiments showed that the joint sensors provided more accurate feedback than motor-based estimation during interaction. Closed-loop experiments further demonstrated improved joint trajectory tracking using joint-sensor feedback and force-controlled object pinching using tactile feedback. The proposed architecture offers a practical path toward sensorized robotic hands for dexterous manipulation.
Read the full abstract ▾
All authors Jaehoon Kim, Boshi An, Lazaros Christoforidis, Michail Papadakis, Victor Kartsch, Robert K. Katzschmann
Robotics
Evidence-Based Landing Site Selection and Vison-Based Landing for UAVs in Unstructured Environments
Sina Sajjadi, Jacopo Panerati, Sina Soleymanpour et al. (+3 more)
Autonomous landing in cluttered or unstructured environments remains a safety-critical challenge for unmanned aerial vehicles (
UAVs), particularly under noisy perception caused by sensor uncertainty and platform-induced disturbances such as vibration. This paper presents an evidence-based probabilistic framework for autonomous
UAV landing that explicitly separates decision-making under uncertainty from execution via visual servoing. Landing safety is modeled as a latent variable and inferred through recursive accumulation of frame-wise visual likelihoods derived from flatness, slope, and obstacle cues, yielding a temporally consistent belief map that is robust to transient perception errors. Physical feasibility is enforced through a hard
geometric constraint based on the minimum required landing radius of the
UAV, ensuring that undersized but visually appealing regions are rejected. The final landing site is selected using constrained maximum a posteriori estimation. Once selected, the
UAV locks onto the target region using ORB feature tracking and performs precise
alignment and descent via image-based visual servoing (IBVS). The proposed approach is validated through both real-world laboratory experiments and high-fidelity simulations in Nvidia Isaac Sim, demonstrating consistent, cautious, and stable landing behavior across domains.
Read the full abstract ▾
All authors Sina Sajjadi, Jacopo Panerati, Sina Soleymanpour, Varunkumar Mehta, Farrokh Janabi-Sharifi, Iraj Mantegh
Robotics
SixthSense: Task-Agnostic Proprioception-Only Whole-Body Wrench Estimation for Humanoids
Xingzhou Chen, Xiayan Xu, Yan Ning et al. (+8 more)
Humanoid robots are entering our physical world at scale, yet as oversized toys–good at singing and dancing, but short on force-interaction capabilities for practical tasks. Bridging this gap necessitates prioritizing reliable contact perception as a fundamental requirement. Estimating external wrenches in
humanoids is complicated by floating-base dynamics and indeterminate contact locations. Existing analytical frameworks require idealistic assumptions and hard-to-obtain measurements, which are often unavailable in practice. To bridge this gap, we propose SixthSense, a task-agnostic approach that infers whole-body contact timing, location, and wrenches from proprioception and IMU data alone. To capture the multi-modal dynamics between unstructured contact inputs and the uncertain motion outputs, we employ conditional flow matching to tokenize proprioceptive histories and estimate a spatiotemporally sparse contact-event flow. SixthSense serves as a plug-and-play perception module for applications including collision detection, physical human-
robot interaction, and force-feedback teleoperation. Experiments across standing, walking, and whole-body motion-tracking policies showcased unprecedented performance in diverse behaviors.
Read the full abstract ▾
All authors Xingzhou Chen, Xiayan Xu, Yan Ning, Jiyu Yu, Yizheng Zhang, Siyi Qian, Lingzhu Xiang, Jiahao Chen, Yuquan Wang, Haodong Zhang, Ling Shi
Robotics · Cross-listed from eess.SY
Quasi-Static Control of Discrete Cosserat Rod
Srishti Siddharth, Domenico Campolo and Ravi Banavar
In this paper, we design feedback control laws for soft
robots modelled using the Cosserat rod theory, which is spatially discretised using the Piecewise Constant Strain (PCS) approach. The PCS approach approximates the nonlinear PDEs describing the Cosserat rod by a finite-dimensional system of nonlinear ODEs. This simplification results in a model describing soft
robots which is similar to the serial rigid-link manipulators. We design feedback control laws for the quasi-static PCS model by using external wrenches as control inputs. The control laws are designed based on feedback linearisation in strain and task spaces. An extensive set of numerical results demonstrates the performance of the control laws for end-effector trajectory tracking and shape control of soft
robots.
Read the full abstract ▾
Robotics
ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescue
Daoxuan Zhang, Ping Chen, Jianyi Zhou et al. (+1 more)
The rapid advancement of
Multimodal Large
Language Models (MLLMs) has empowered Unmanned Aerial Vehicle (
UAV) with exceptional capabilities in spatial
reasoning, semantic understanding, and complex decision-making, making them inherently suited for
UAV Search and Rescue (SAR). However, existing
UAV SAR research is dominated by traditional
vision and path-planning methods and lacks a comprehensive and unified
benchmark for
embodied agents. To bridge this gap, we first propose the novel task of
Embodied Search and Rescue (ESAR), which requires aerial
agents to autonomously explore complex environments, identify rescue clues, and reason about victim locations to execute informed decision-making. Additionally, we present ESARBench, the first comprehensive
benchmark designed to evaluate MLLM-driven
UAV agents in highly realistic SAR scenarios. Leveraging Unreal Engine 5 and AirSim, we construct four high-fidelity, large-scale open environments mapped directly from real-world Geographic Information System (GIS) data to ensure photorealistic landscapes. To rigorously simulate actual rescue operations, our
benchmark incorporates dynamic variables including weather conditions, time of day, and
stochastic clue placement. Furthermore, we create a
dataset of 600 tasks modeled after real-world rescue cases and propose a robust set of evaluation metrics. We evaluate diverse baselines, ranging from traditional heuristics to advanced ground and aerial MLLM-based ObjectNav
agents. Experimental results highlight the challenges in ESAR, revealing critical bottlenecks in spatial
memory, aerial adaptation, and the trade-off between search efficiency and flight safety. We hope ESARBench serves as a valuable resource to advance research on
Embodied Search and Rescue domain. Source code and project page: https://4amgodvzx.github.io/ESAR.github.io.
Read the full abstract ▾
All authors Daoxuan Zhang, Ping Chen, Jianyi Zhou, Shuo Yang
Robotics
Assistance Without Interruption: A Benchmark and LLM-based Framework for Non-Intrusive Human-Robot Assistance
Yuedi Zhang, Shuanghao Bai, Wanqi Zhou et al. (+4 more)
Human-
robot interaction (HRI) has long studied how
agents and people coordinate to achieve shared goals. In this work, we formalize and
benchmark the non-intrusive assistance as an independent paradigm of HRI, where a
robot proactively supports a human's ongoing multi-step activities while strictly avoiding interruptions. Unlike conventional HRI tasks that rely on direct commands, explicit negotiation, or proactive interventions based on user habits and history, our task treats the human's plan as the primary process and formulates assistance as a joint decision over when to act and what to do. To systematically evaluate this problem, we establish a simulation
benchmark, NIABench, along with new metrics tailored to the non-intrusive assistance task. We further propose a hybrid architecture that integrates an LLM with a scoring model. The scoring model first applies semantic
retrieval to prune large candidate action sets, and then a ranker evaluates human-step and
robot-action pairs, enabling
reasoning over timing and cross-step dependencies. Comprehensive experiments on both NIABench and real-world scenarios demonstrate that our method achieves proactive, non-intrusive assistance that reduces human effort while preserving task effectiveness.
Read the full abstract ▾
All authors Yuedi Zhang, Shuanghao Bai, Wanqi Zhou, Haoran Zhang, Qi Zhang, Zhirong Luan, Badong Chen
Robotics · Cross-listed from cs.CV
VoxAfford: Multi-Scale Voxel-Token Fusion for Open-Vocabulary 3D Affordance Detection
Haowen Sun, Shaolong Zhang, Mingyang Li et al. (+6 more)
Open-vocabulary 3D affordance detection requires localizing interaction regions on point clouds given novel affordance descriptions. Recent methods extend
multimodal large
language models (MLLMs) with special output
tokens that are decoded into
segmentation masks. However, these
tokens are produced through autoregressive generation, which models sequential dependencies rather than spatial neighborhood relations, leaving them semantically rich but spatially impoverished for 3D localization. We propose Voxel-enhanced Affordance detection (VoxAfford), which bypasses this bottleneck by injecting multi-scale
geometric features from a frozen pre-trained 3D VQVAE encoder into the output
tokens after generation. Each output
token uses its affordance semantics as a query to retrieve relevant
geometric patterns from its paired voxel scale via cross-
attention, with a learned compatibility gate controlling the injection strength. The enhanced
tokens are then aggregated into a spatially-aware affordance
prompt through semantic-conditioned
attention and propagated alongside per-point features to generate the final mask. Experiments on open-vocabulary affordance detection tasks show that VoxAfford achieves state-of-the-art performance with approximately an 8% improvement in mIoU, and real
robot experiments confirm zero-shot transfer to novel objects.
Read the full abstract ▾
All authors Haowen Sun, Shaolong Zhang, Mingyang Li, Chengzhong Ma, Xinzhe Chen, Qiongjie Cui, Xingyu Chen, Zeyang Liu, Xuguang Lan
Robotics
Terrain Perception for Agricultural UAVs in Complex Farmland via Rotating mmWave Radar
Zhihao Zhan, Le Tao, Shaobin Li et al. (+5 more)
Accurate terrain perception is essential for terrain-following flight of agricultural unmanned aerial vehicles (
UAVs), yet remains challenging in real-world farmland due to occlusions, complex terrain geometry, and environmental disturbances. Millimeter-wave (mmWave) radar is a promising sensing modality for this task due to its robustness to adverse conditions; however, existing
UAV-mounted radar systems rely on fixed field of view (FoV) and terrain extraction methods designed for dense
LiDAR data, leading to incomplete and unreliable terrain estimation. To address these limitations, we present a low-cost rotating mmWave radar-enabled terrain perception framework for agricultural
UAVs operating in complex farmland environments. Specifically, a mechanically rotating sensing design is introduced to enlarge spatial coverage and improve terrain observability beyond the limitations of fixed-view radar under dynamic low-altitude flight. Building upon this sensing capability, we further design a pose-consistent terrain reconstruction pipeline tailored for sparse, noisy, and partially observable radar data, enabling reliable ground extraction and continuous terrain surface estimation in challenging agricultural scenarios. The complete system is deployed on a real agricultural
UAV platform and comprehensively evaluated through extensive field experiments. Experimental results demonstrate improved terrain coverage and estimation accuracy, achieving an F1 score of 94.42 for ground
segmentation, while the closest rival only achieves 90.48. Thus, leading to more robust terrain following flight.
Read the full abstract ▾
All authors Zhihao Zhan, Le Tao, Shaobin Li, Chenxin Fang, Xingrui Yang, Liang Li, Rui Fan, Yuhang Ming
Robotics
Bi-Level Reinforcement Learning Control for an Underactuated Blimp via Center-of-Mass Reconfiguration
Xiaorui Wang, Hongwu Wang, Yue Fan et al. (+2 more)
This paper investigates goal-directed tracking control of underactuated blimps with center-of-mass (CoM) reconfiguration. Unlike conventional overactuated blimp designs that rely on redundant actuation for simplified control, this paper focuses on a compact architecture consisting of two thrusters and a movable internal slider, aiming to improve energy efficiency and payload capacity. This hardware-efficient configuration introduces significant underactuation and strong nonlinear coupling between CoM dynamics and vehicle motion. To address these challenges, this paper proposes a bi-level reinforcement learning framework that explicitly decouples task-level CoM planning from continuous thrust control. The outer policy determines a target-dependent CoM configuration prior to flight, while the inner policy generates thrust commands to track straight-line references. To ensure stable learning, this paper introduces a two-stage learning strategy, supported by a convergence analysis of the resulting bi-level process. Extensive simulations and real-world experiments on a 27-goal evaluation set demonstrate that the proposed method consistently outperforms fixed-CoM baselines and PID-based controllers, achieving higher tracking accuracy, enhanced robustness, and reliable sim-to-real transfer.
Read the full abstract ▾
All authors Xiaorui Wang, Hongwu Wang, Yue Fan, Hao Cheng, Feitian Zhang
Robotics · Cross-listed from cs.HC
What Does a Meow Mean? In Search of Intuitively Understandable Communication by a Nonverbal Companion Robot
Vivienne Bihe Chi, Claudia B. Rébola and Bertram F. Malle
Older adults living alone have a number of challenges, and
robots can help with some of them–by providing reminders, initiating activity, or offering comfort. As part of developing a cat
robot with limited assistive functions, we designed a set of nonverbal communication signals, both auditory (cat sounds) and visual (icons on a small display). To evaluate these signals we used a mixed-methods, user-centered approach. After a pilot study, a focus group with older adults suggested revisions to the initial signal set. A large-sample online experiment then tested whether adults over the age of 65 could accurately infer the
robot's communicative intentions. When both visual and auditory signals were present, accuracy was high. When visual signals were absent, accuracy often decreased; when auditory signals were absent, accuracy sometimes increased. So the auditory signals were less helpful, except when the
robot conveyed strong sentiments (e.g., purring while being petted).
Read the full abstract ▾
Robotics
A Principled Approach for Creating High-fidelity Synthetic Demonstrations for Imitation Learning
Moniruzzaman Akash and Momotaz Begum
Recent advances in 3D Gaussian Splatting (3DGS) have enabled visually realistic demonstration generation from a single expert trajectory and a short multi-view scan. However, existing 3DGS-based synthesis pipelines typically generate new motions using sampling-based planners or trajectory optimization, which often deviate substantially from the expert's demonstrated path. While such deviations may be acceptable for tasks insensitive to motion shape, they discard subtle spatial and temporal structure that is critical for contact-rich and shape-sensitive manipulation, causing increased demonstration diversity to harm downstream policy learning. We argue that demonstration synthesis should treat the expert trajectory as a strong prior. Building on this principle, we propose a framework that synthesizes diverse task demonstrations while explicitly preserving expert motion structure. We model the expert trajectory using Dynamic Movement Primitives (DMPs) and retarget it to new goals, object configurations, and viewpoints within a reconstructed 3DGS scene, yielding phase-consistent, shape-preserving motion by construction. To safely realize this expert-preserving diversity in cluttered scenes, we introduce an analytic obstacle-aware DMP formulation that operates directly on the continuous density field induced by the 3DGS representation. This enables collision avoidance while minimally perturbing the nominal expert motion, unifying photorealistic rendering and
geometric reasoning without additional scene representations. We evaluate our approach on a Spot mobile manipulator across three manipulation tasks with increasing sensitivity to trajectory fidelity. Compared to planner- and optimization-based synthesis, our method produces trajectories with lower deviation and collision rates and yields higher task success when training
diffusion-based visuomotor policies.
Read the full abstract ▾
Robotics
Dynamics Aware Quadrupedal Locomotion via Intrinsic Dynamics Head
Aman Arora and Nalini Ratha
Quadrupedal locomotion plays a critical role in enabling agile, versatile movement across complex terrains. Understanding and estimating the underlying physical dynamics are essential for achieving efficient and stable quadrupedal locomotion. We propose a novel training framework for quadrupedal locomotion that enables the Control Policy to understand and reason about physical dynamics. In simulation, we concurrently train an Intrinsic Dynamics (ID) Head that learns state-to-torque dynamics alongside the Control Policy, and we define a dynamics reward enabled by the ID Head that encourages the Policy toward more predictable dynamical behavior. We also provide a mechanism to tune the learned dynamics in the resulting Policy by controlling the training coefficients of the ID Head. Our simulation experiments show that this mechanism drives convergence to better optima across a wide range of standard quadrupedal locomotion rewards, yielding more efficient and smoother policies. Our real-
robot experiments demonstrate sim-to-real transfer of these improvements, with significant gains in torque efficiency (16.8%), action rate (18.6%), and mechanical power (12.8%), while improving safe torque occupancy by 6.4%.
Read the full abstract ▾
Robotics
To Do or Not to Do: Ensuring the Safety of Visuomotor Policies Learned from Demonstrations
Riad Ahmed, Moniruzzaman Akash and Momotaz Begum
Task success has historically been the primary measure of policy performance in imitation learning (IL) research. This characteristics strictly limits the ubiquitous applications of IL
algorithms in field robotics where safety assurance, in addition to task-success, is of paramount importance. It is often desirable for an IL-powered
robot in the field not to roll out a policy, and hence score a poor performance, if the safety is not guaranteed. Although this trade-off between safety and performance is well investigated in classical control literature, policy safety is a heavily underexplored domain in IL research. There is no universal definition of safety in IL. To make things worst, many existing theoretical works on safety is notoriously difficult to extend to IL-powered
robots in the field. This paper offers important insights on the safety and performance of IL policies. We propose execution guarantee, a policy-agnostic safety measure that guarantees the maximum task success for a visuomotor IL policy, despite minor run-time changes, from within a specific region in the state space. We leverage recent advances in view synthesis to identify such regions in the state space for an IL policy and explore a fundamental result on set invariance - namely, Nagumo's sub-tangentiality condition - to prove and operationalize execution guarantee from inside that region. Experiments with a Franka
robot, both in simulation and real world, demonstrate how the proposed safety analysis allows various IL policies to achieve maximum task success with guarantee. We also demonstrate some interesting results on how a recovery policy - a by-product of the proposed safety analysis - can help to increase the policy performance and thereby mitigating the safety-performance tradeoff in IL.
Read the full abstract ▾
Robotics
TAIL-Safe: Task-Agnostic Safety Monitoring for Imitation Learning Policies
Riad Ahmed and Momotaz Begum
Recent imitation learning (IL)
algorithms such as flow-matching and
diffusion policies demonstrate remarkable performance in learning complex manipulation tasks. However, these policies often fail even when operating within their training distribution due to extreme sensitivity to initial conditions and irreducible approximation errors that lead to compounding drift. This makes it unsafe to deploy IL policies in the field where out-of-distribution scenarios are prevalent. A prerequisite for safe deployment is enabling the policy to determine whether it can execute a task the way it was learned from demonstrations. This paper presents TAIL-Safe, a principled approach to identify, for a trained IL policy, a safe set from where the policy empirically succeeds in completing the learned task. We propose a Lipschitz-continuous Q-value function that maps state-action pairs to a long-term safety score based on three short-term task-agnostic criteria: visibility, recognizability, and graspability. The zero-superlevel set of this function characterizes an empirical control invariant set over state-action pairs. When the nominal policy proposes an action outside this set, we apply a recovery mechanism inspired by Nagumo's
theorem that uses gradient ascent to the Q-function to steer the policy back to safety. To learn this Q-function, we construct a high-fidelity digital twin using Gaussian Splatting that enables systematic collection of failure data without risk to physical hardware. Experiments with a Franka Emika
robot demonstrate that flow-matching policies, which fail under run-time perturbations, achieve consistent task success when guided by the proposed TAIL-Safe.
Read the full abstract ▾
Robotics
VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model
Wenhao Li, Xiu Su, Yichao Cao et al. (+5 more)
Vision-
Language-Action (VLA) models have demonstrated remarkable capabilities and generalization in
embodied manipulation. However, their decision-making relies on a fast, instinctive process that lacks deliberation. This strategy often leads to suboptimal or catastrophic actions when facing complex or ambiguous scenarios that require greater consideration. In this paper, we introduce VLA-ATTC, a framework that endows VLA models with adaptive test-time compute (TTC). VLA-ATTC employs an uncertainty-based “cognitive clutch” to dynamically transition from reflexive execution to a TTC deliberation phase when necessary. During TTC phase, a novel Relative Action Critic (RAC) model identifies the optimal action from generated candidates via pairwise comparisons. This relative mechanism replaces unstable absolute value estimation, significantly simplifying the learning objective. Furthermore, we introduce an efficient sampling strategy to amortize computational costs and an automated data pipeline that curates preference pairs without manual annotation. On the LIBERO-LONG
benchmark, VLA-ATTC reduces the failure rate of the SOTA model PI0.5 by over 50%. We will open-source all the code and weights.
Read the full abstract ▾
All authors Wenhao Li, Xiu Su, Yichao Cao, Hongyan Xu, Xiaobo Xia, Shan You, Yi Chen, Chang Xu
Robotics
Sentinel-VLA: A Metacognitive VLA Model with Active Status Monitoring for Dynamic Reasoning and Error Recovery
Wenhao Li, Xiu Su, Dan Niu et al. (+6 more)
Vision-
language-action (VLA) models have advanced the field of
embodied manipulation by harnessing broad world knowledge and strong generalization. However, current VLA models still face several key challenges, including limited
reasoning capability, lack of status monitoring, and difficulty in self-correction. In this paper, we introduce Sentinel-VLA, a metacognitive VLA model equipped with an active “sentinel” module to monitor real-time execution status. Only when necessary, such as during initial planning or upon detecting an error, the model triggers a dynamic
reasoning or formulate error recovery solutions. This on-demand
reasoning mechanism ensures robust decision-making while minimizing computational overhead. Notably, all training data (spanning 44 tasks and over 2.6 million transitions) is automatically generated and annotated through our designed pipeline. We also propose the Self-Evolving Continual Learning (SECL)
algorithm, which allows Sentinel-VLA to identify its capability boundaries and automatically collect data for expansion, paired with Orthogonal Continual Adapter (OC-Adapter) to constrain parameter updates to an orthogonal space, thereby preventing catastrophic forgetting. Real-world experiments demonstrate that Sentinel-VLA boosts the task success rate by over 30% compared to the SOTA model, PI0. We will open-source all the code, weights, and data generation pipeline.
Read the full abstract ▾
All authors Wenhao Li, Xiu Su, Dan Niu, Yichao Cao, Hongyan Xu, Zhe Qu, Lei Fan, Shan You, Chang Xu
Robotics · Cross-listed from physics.app-ph
A skin-like conformal sensor for real-time shape mapping
Kaiping Yin, Sooik Im, Chaorui Qiu et al. (+5 more)
Reliable real-time 3D shape sensing is essential for robust control and interpretation of deformable systems during motion. Existing
vision-based approaches require line-of-sight and complex instrumentation, limiting operation in occluded and space-constrained settings. Here, we introduce a scalable, skin-like sensor that reconstructs its continuous 3D deformation in real time from distributed strain measurements. The device embeds a 2D array of mirror-stacked, printed oxidized eutectic gallium-indium (o-EGaIn) strain gauges within an elastomeric film to measure off-neutral-axis strains. Combined with a mechanics-informed observation model and a fast optimization routine, the system estimates local
curvature, elongation, offset, and orientation under concurrent stretching, bending, and indentation, enabling reconstruction of complex surfaces. A 5-by-5 array with a 12 mm pitch achieves a mean surface reconstruction error of 0.62 mm with 0.1s latency across all tested scenarios. When conforming to complex surfaces, the sensor provides fast 3D shape mapping of the underlying geometry. Demonstrations involving palm gesturing, finger indentation, and contact-induced balloon deformation highlight utility for epidermal motion tracking, haptic interaction, and intraoperative monitoring.
Read the full abstract ▾
All authors Kaiping Yin, Sooik Im, Chaorui Qiu, Yun Bai, Xiangyu Lu, Chenhang Li, Junjie Yao, Xiaoyue Ni