Robotics
Leveraging Cost-effective Robotics for K-12 STEM Education through Water Quality Monitoring Tasks
Rishi Mukherjee, Andrew Ruiz, Travis Henderson et al. (+6 more)
Engaging K-12 students in authentic scientific research remains a significant challenge, particularly at the intersection of environmental science and robotics. We introduce the Jar Jar ROV, a low-cost, open-source Remotely Operated Vehicle (ROV) platform designed for citizen science-based water quality monitoring by middle school students. This paper presents the design of the platform and the results of a large-scale deployment with over 100 students across a US state who built, programmed, and deployed the ROVs in local lakes. The educational framework yielded high student engagement in hands-on activities, with ROV construction earning a perfect average score from mentors. Scientifically, the program established a grassroots monitoring network, generating nearly eleven thousand validated measurements of temperature, pH, dissolved oxygen, and turbidity. However, our evaluation identified a critical "engagement gap," with student interest declining sharply during more complex tasks such as electronics assembly and data uploading. This paper contributes both a validated, scalable model for integrating robotics into environmental education and a data-driven roadmap for future improvements. These enhancements focus on lowering technical barriers and creating a more intuitive link between data collection and scientific discovery, addressing a key challenge in empowering the next generation of citizen scientists.
Read the full abstract ▾
All authors Rishi Mukherjee, Andrew Ruiz, Travis Henderson, Resha Tejpaul, Kris Simonson, David Mulla, Brian McNeil, Nikolaos Papanikolopoulos, Junaed Sattar
Robotics
Online Target-less Radar-LiDAR-Camera Extrinsic Calibration via Joint Optimization
Gunhee Shin, Yunsoo Kim, Chanhyuk Lee et al. (+7 more)
Fusing radar,
LiDAR, and camera enables robust perception in diverse and adverse conditions, but the fusion performance critically depends on accurate extrinsic calibration among the three sensors. In this paper, we address the problem of online target-less extrinsic calibration for the radar-
LiDAR-camera system. Existing target-less methods are mostly designed for a single sensor pair, and composing the pairwise results does not guarantee consistency across the three sensors. Moreover, the sparse and noisy radar measurements make the radar-involving pairs unreliable. To tackle these challenges, we propose a joint calibration framework that constructs residuals for each sensor pair and optimizes the extrinsics of all pairs together to minimize the overall residual. Furthermore, we introduce an adaptive radar noise filter that rejects spurious radar returns using a range-dependent margin, and a correspondence accumulation strategy that aggregates sparse radar correspondences over frames. We validate our method on an in-house radar-
LiDAR-camera
dataset covering diverse urban environments, where it reduces calibration errors across all sensor pairs over a state-of-the-art camera-
LiDAR baseline.
Read the full abstract ▾
All authors Gunhee Shin, Yunsoo Kim, Chanhyuk Lee, Wanhee Kim, Minwoo Lee, Sungwoo Han, Jeongwoo Woo, Hyuntai Chin, Minha Park, Hyun Myung
Robotics
NM-LIO: Multiple LiDAR-Inertial Odometry Addressing LiDAR Measurement Noise Discrepancy
Gunhee Shin, Seungjae Lee, Minho Oh et al. (+4 more)
Multiple
LiDAR-inertial odometry methods have been widely applied in robotic applications owing to their enhanced accuracy and reliability. However, adopting multiple
LiDAR systems can be challenging due to the discrepancies in measurement noise across different
LiDARs. Existing methods have typically overlooked the noise discrepancies, which can significantly affect accuracy. In this paper, we propose Noise-aware Multiple
LiDAR-Inertial Odometry (NM-LIO) that addresses the noise discrepancies. We integrate a noise model to quantify the measurement noise of each
LiDAR. Additionally, we estimate the uncertainty of the residuals based on the measurement noise, allowing the measurement model to capture the noise discrepancies. Our proposed method is evaluated on a public multiple
LiDAR dataset and compared with state-of-the-art methods. The experimental results demonstrate that the proposed method can accurately estimate the odometry in various environments by accounting for the noise discrepancies.
Read the full abstract ▾
All authors Gunhee Shin, Seungjae Lee, Minho Oh, Dongkyu Lee, Jaeyoung Lee, Youngwoo Seo, Hyun Myung
Robotics · Cross-listed from cs.MA
Nonlinear Density-Driven Optimal Control (D2OC) for Multi-Agent Spatial Coverage via Sequential Convex Programming
Julian Martinez and Kooktae Lee
This paper presents a nonlinear extension of Density-Driven Optimal Control (D2OC) for multi-
agent spatial coverage with prescribed density distributions. Rather than assigning individual target locations, D2OC drives the collective spatial distribution of
agents toward a desired density through a Wasserstein-based objective. We extend this framework to multi-step finite-horizon control for discrete-time control-affine nonlinear systems using sequential convex programming. At each control update, the nonlinear dynamics are locally linearized over the prediction horizon, yielding a strictly convex quadratic program that preserves the Wasserstein barycentric structure while directly incorporating input constraints. We further characterize the effect of constrained control deviations and nonlinear Taylor remainders on the accuracy of the local linear prediction, establishing an explicit finite-horizon error bound and a two-step specialization for receding-horizon implementation. The resulting method retains the decentralized, distribution-driven nature of D2OC while providing a computationally efficient optimization procedure for nonlinear multi-
agent systems. Simulations with unicycle and quadrotor teams show coverage performance comparable to nonlinear model predictive control, while substantially reducing computation time. These results demonstrate a tractable and theoretically characterized framework for density-driven spatial coverage under nonlinear dynamics.
Read the full abstract ▾
Robotics
DreamFormer: Dream Imitation with a Transformer World Model for Language-Conditioned Robotic Manipulation
Mostafa Kotb, Cornelius Weber, Muhammad Burhan Hafez et al. (+1 more)
We introduce DreamFormer, a model-based
agent that acquires
language-conditioned, multi-task skills by imitating expert demonstrations within the latent imagination of a learned world model. DreamFormer first learns a task-agnostic
Transformer world model from unstructured play data, then acquires task-specific behaviors by optimizing an intrinsic reward that aligns
agent-generated rollouts with expert demonstrations in latent space. Since the policy is trained on-policy inside imagination, it is exposed to its own errors during training, mitigating the
covariate shift inherent to offline behavioral cloning. To make long-horizon imagination affordable, DreamFormer encodes a high-resolution multi-view robotic observation into a single input
token, avoiding both spatial downsampling and the multi-
token representations used by prior
Transformer world models. On the long-horizon CALVIN
benchmark, DreamFormer outperforms LUMOS, the comparable model-based
agent, on single-environment evaluation (2.52 vs 2.34 average tasks completed per chain of five) while keeping imagination rollouts tractable. Against HULC, the behavior cloning baseline, it nearly doubles performance on zero-shot transfer to an unseen environment (1.30 vs 0.67), indicating that dynamics learned by the world model transfer more readily than a directly cloned policy. This is consistent with the role attributed to internal models in biological
agents, where a model of environment dynamics supports behavior in situations not previously encountered.
Read the full abstract ▾
All authors Mostafa Kotb, Cornelius Weber, Muhammad Burhan Hafez, Stefan Wermter
Robotics
WasserMan: Benchmark for Underwater Manipulation Policy Learning
Danil Belov, Artem Erkhov, Sergei Parsegov et al. (+1 more)
Underwater manipulation couples visual decisions, contact forces and a thruster-controlled floating base. We introduce WasserMan, to our knowledge the first multi-task simulation
benchmark for visuomotor learning of floating-base underwater contact manipulation. It provides ten expert-solvable tasks, two vehicle-arm platforms, a bimanual configuration and underwater dynamics. Nine tasks have learned-policy evaluations. We compare ACT,
diffusion policies (DP) and behavioral cloning on six tasks, with three training runs and equal sampled-window budgets. Pretrained SmolVLA adds evaluations on three tasks. In the tested settings, removing integral action can prevent completion, while changing action interfaces can reduce learned-policy success despite successful expert replay. Currents produce task-dependent changes in completion and actuation effort. Versioned tasks, demonstrations and per-episode evidence support a protocol separating expert feasibility, learned completion and execution effort.
Read the full abstract ▾
All authors Danil Belov, Artem Erkhov, Sergei Parsegov, Pavel Osinenko
Robotics · Cross-listed from cs.AI
RAGrasp: Geometry-Semantic Template Retrieval and Grasp Transfer
Shenzhe Zhu, Chengxiao He and Jan Harder
We present RAGrasp, a
retrieval-augmented pipeline for planar parallel-jaw grasping from a compact set of locally collected, grasp-annotated RGB-D (color and depth) templates. Unlike task-specific predictors trained primarily on large public or synthetic grasp
datasets, RAGrasp requires no end-to-end retraining for a new deployment.Its template
memory is constructed from observations collected with the deployment camera,
robot, and gripper in the target workspace, thereby aligning stored examples with the local sensing and embodiment conditions. The system uses self-supervised DINOv2 visual fea- tures together with appearance and depth cues to
prompt the Segment Anything Model 2 (SAM2), which isolates the query object. A two-stage geometry-semantic
retrieval cascade then selects a template, and a confidence gate chooses one of two grasp- transfer
estimators. The transferred grasp is refined using mask- support and silhouette-contact constraints before calibrated 2D- to-3D conversion. In real-world trials, RAGrasp achieves 20/20 successful grasps on seen objects and 19/20 on unseen objects. Within the evaluated setting, the results demonstrate deployment- specific grasp adaptation from limited local annotation and tolerance to the tested viewpoint and illumination changes.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
Jinzhou Tang, Zijun Zhang, Jing Yang et al. (+10 more)
Building interactive simulators from real-world observations is a promising way to scale embodied data, but current pipelines still rely heavily on manual environment construction and calibration. We study whether frontier foundation models and coding
agents can automate this process end to end. We formulate autonomous
video-to-simulation as a software engineering task in which an
agent observes an embodied
video, constructs the corresponding simulated environment and
robot behavior, and iteratively refines the result through execution feedback. To evaluate this capability, we introduce
Video2World, a
benchmark comprising 222 reconstruction instances derived from 189
robot and human demonstration
videos.
Video2World measures reconstructed worlds along
geometric fidelity, dynamic fidelity, and functional correctness, capturing spatial perception, physical
reasoning, and executable interaction. Evaluating 9 frontier coding-
agent systems reveals a sharp improvement in Task success beginning with Claude Opus 5, rising from below 5% to over 15%, while substantial gaps to human-assisted reconstruction remain. We further find that worlds that look better could work worse: better visual fidelity does not always lead to higher task success. This echoes the broader gap between perceptual realism and factual correctness observed in generative models.
Read the full abstract ▾
All authors Jinzhou Tang, Zijun Zhang, Jing Yang, Yuchen Yan, Kun Zhou, Lingjun Mao, Ruobing Han, Jinglin Cao, Wenpeng Xu, Lukun He, Minghao Fu, Fan Feng, Biwei Huang
Robotics
Reachability-Guided Sequential Quadratic Programming-Guarded Model Predictive Path Integral for Safe Nonlinear Predictive Control
Alexandre Didier, Jason J. Choi, Namhoon Cho et al. (+2 more)
Safe
robot control often requires combining long-horizon performance optimization with hard state and input constraints, but existing approaches tend to address this tradeoff partially. Sampling-based model predictive control (MPC) methods such as model predictive path integral (MPPI) are effective in handling nonlinear and nonconvex environments, yet their finite-sample rollouts and unconstrained weighted-average update can return an unsafe control. Deterministic nonlinear MPC can explicitly incorporate constraints, but its real-time safety and performance depends strongly on warm starts and local convergence. Hamilton–Jacobi reachability (HJR) provides rigorous safety certificates, but offline value-function computation remains practical only for reduced-order models. We propose ReSQ-MPPI, a reachability-informed generation–refinement architecture that combines these complementary strengths. An HJR value function computed offline for a reduced-order model guides online MPPI sampling toward safe, promising trajectory candidates. The MPPI solution is then refined by a small number of sequential quadratic programming (SQP) iterations in a full-order MPC problem. A key observation is that the standard MPPI inference step is an unconstrained weighted least-squares problem; ReSQ-MPPI replaces it with a constrained MPC refinement that recovers the MPPI update when it is feasible and minimally modifies it otherwise. Simulations in cluttered navigation and autonomous racing environments demonstrate that ReSQ-MPPI improves safety and performance over standalone MPPI, MPC, and reachability-filtered sampling-based control baselines.
Read the full abstract ▾
All authors Alexandre Didier, Jason J. Choi, Namhoon Cho, Claire J. Tomlin, Melanie N. Zeilinger
Robotics · Cross-listed from cs.DC
Budget-Constrained Fault-Tolerant Mutual Visibility for Autonomous Robots under the Mobility Fault Model
Prakhar Shukla, Animesh Maiti, Shivam Kumar et al. (+1 more)
We investigate the mutual visibility problem for a swarm of $n\ge3$ autonomous mobile
robots under the budget-constrained mobility fault model. The
robots are opaque, so if three
robots are collinear, the middle
robot obstructs the visibility between the other two. Each
robot is assigned a finite movement budget, reflecting its limited energy, that bounds the total distance it may traverse during the execution. Moreover, an arbitrary number of
robots may become permanently immobile due to mobility faults. The objective is to design a distributed
algorithm that enables the non-faulty
robots to coordinate their movements so that, within a finite time, every non-faulty
robot attains unobstructed visibility of all
robots in the system, including the faulty ones, while respecting the prescribed movement budget. We consider luminous
robots operating under the $\mathsf{SSYNC}$ model with non-rigid movements, without any agreement on their local coordinate systems, and equipped only with a common fixed reference point. We present a deterministic distributed
algorithm that solves the problem despite an arbitrary number of mobility faults. The
algorithm guarantees mutual visibility for the non-faulty
robots, respects the movement budget of every
robot, provides collision-free movements for the
robots, and uses only 12 light colors.
Read the full abstract ▾
All authors Prakhar Shukla, Animesh Maiti, Shivam Kumar, Subhash Bhagat
Robotics
AgenticTactileVLA: Contact-Guided Execution-Time Supervision for Generalizable Dexterous Manipulation without VLA Retraining
Elizaveta Semenyakina, Ivan Snegirev, Mikhail Kiselev et al. (+4 more)
Vision-
language-action policies may predict a transferable manipulation strategy yet fail to realize it reliably on the encountered object: objects compatible with the same grasp differ in geometry and compliance, and visual feedback degrades under closure occlusion. AgenticTactileVLA is presented as an execution-time supervisor that shifts part of object-specific adaptation from prediction to physical interaction. A fixed VLA provides the approach and hand targets; the supervisor decides whether to remain transparent, refine finger flexion, retain or release the corrected configuration, return control to the VLA for retry, or select a compliant hand-control regime. It uses finger-position and motor-effort feedback as proprioceptive contact evidence and requires neither tactile sensors nor VLA retraining. On a Unitree G1 with a BrainCo Revo2 hand, a randomized matched-block evaluation on five objects held out from VLA training yields 61.3% completion for the base VLA, 72.0% for unconditional close-to-stall control, and 84.0% for the supervisor under a shared budget; the gain is positive on every object and persists under moderate pose perturbations. Ablations show the gain is not explained by extended closure alone, and that selective triggering reduces correction episodes by 65.7% with no detected change in completion. A retention audit shows acceptance predicts retention in 88.9% of held-out cases, while compliant objects expose conservative false rejection. A thin-walled-cup study demonstrates contextual routing to compliant control, matching an always-compliant reference. These results suggest that contact-guided execution-time adaptation can improve the object-level generalization of a fixed VLA to held-out objects by adapting physical realization without object-specific retraining.
Read the full abstract ▾
All authors Elizaveta Semenyakina, Ivan Snegirev, Mikhail Kiselev, Miguel Altamirano Cabrera, Artem Lykov, Hajira Amjad, Dzmitry Tsetserukou
Robotics
Frame-Level Temporal Alignment for Human-to-Robot Visual Adaptation
Xizhe Zhang, Jingfeng Zhang, Zirun Zhou et al. (+1 more)
Transferring visual representations pretrained on human
videos to
robot manipulation requires learning reliable correspondences between human and
robot demonstrations. However, paired demonstrations can differ in execution rate and in the proportion of non-key frames that do not directly reflect task progress. Frames at the same relative timestamp may therefore represent different task stages, which can cause correspondence learning to fail. To address these issues, we propose Frame-Level Temporal
Alignment (FLTA), a framework that uses two temporal priors to adapt visual encoders pretrained on human
videos for
robot manipulation. It learns shared task-progress representations without frame-level correspondence annotations, allowing frames at different relative temporal positions to match. A global progress prior combines normalized temporal positions with visual similarity to construct a soft correspondence target. A local temporal order prior penalizes backward transitions while allowing stays and varying forward rates to accommodate execution-rate differences. With ResNet-50 and ViT encoders, our method achieves relative improvements of 46.93% and 65.96%, respectively, over the best baselines in average simulation success rates and achieves higher task success rates on real-world manipulation tasks. These results also suggest that effective human-
robot adaptation depends less on the number of parameters updated than on which parameters are selected. Our project page is available at https://rtx5090ultra.github.io/FLTA-Project-Page/.
Read the full abstract ▾
All authors Xizhe Zhang, Jingfeng Zhang, Zirun Zhou, Hong Jia
Robotics
Human Behavior-Informed Crash Scenario Generation with Real-World Crash Priors for Autonomous Vehicle Safety Evaluation
Mingxing Peng, Xusen Guo, Long Chen et al. (+3 more)
Reliable safety evaluation of autonomous vehicles (AVs) is essential to improving road safety, yet it depends critically on realistic simulation of rare crashes. Existing crash scenario generation methods can increase collision occurrence, but often fail to realistically reproduce how crashes evolve before impact or the distribution of crash types observed in the real world. Here, we present CrashSim, a human behavior-informed crash scenario generation framework that uses real-world crash priors to guide generative multi-
agent traffic simulation for more reliable AV safety evaluation. These priors capture how real-world crashes evolve before impact and how different crash types are distributed, allowing limited crash data to guide realistic and scalable scenario generation across naturalistic driving contexts. We evaluate CrashSim against competing methods, showing that it more closely reproduces real-world pre-impact behavior, collision dynamics, collision geometry and crash-type distributions. We further use CrashSim to construct nuCrash
dataset, containing over 4,000 crash and near-crash scenarios. Closed-loop evaluation of five AV planners shows that nuCrash more effectively exposes differences in planner safety capabilities than nuScenes. An LLM-assisted evaluation
agent further analyzes planner failures to provide capability-level diagnoses and targeted improvement guidance. Together, CrashSim enables realistic and scalable crash generation for more informative AV safety evaluation.
Read the full abstract ▾
All authors Mingxing Peng, Xusen Guo, Long Chen, Xintao Yan, Siyu Teng, Jun Ma
Robotics · Cross-listed from cs.LG
A Bird's-Eye View of Iterative Reward Design
Logan Mondal Bhamidipaty, Lauren Robson, Linda Petrini et al. (+2 more)
Designing effective reward functions in RL typically requires substantial expertise and trial and error. Recent work automates this process with LLM-based systems that generate and iteratively improve reward code using policy feedback. However, these methods are often hard to compare because they differ in implementation details, feedback assumptions, and evaluation environments. To address this, we introduce a
Benchmark for Iterative Reward Design (BIRD) that expresses existing methods in a unified configuration and evaluation space. This lets us compare
algorithms directly, ablate individual design choices, and prototype new components under matched feedback conditions and policy-training budgets. Across MuJoCo, Meta-World, Assistax, and HumanoidBench, we identify a small set of simple design choices that consistently improve performance. Combining these choices yields significantly better performance than the evaluated methods from prior work. Our results highlight the strength of simple baselines and motivate further study of when additional algorithmic complexity improves iterative reward design. Code is available at https://github.com/safety-research/bird.
Read the full abstract ▾
All authors Logan Mondal Bhamidipaty, Lauren Robson, Linda Petrini, Shengrui Lyu, Kamal Ndousse
Robotics
TacOT: Learning Contact-Rich Dexterous Manipulation from Human Demonstrations via Tactile-Guided Optimal Transport
Xingting Li, Yifan Han, Zijian Lin et al. (+4 more)
Learning contact-rich dexterous manipulation from human demonstrations provides a scalable source of interaction data, yet transferring such skills to
robots remains challenging due to unreliable human–
robot correspondence. Existing human-to-
robot transfer methods typically rely on visual appearance or motion similarity, which may associate similar motions with different contact states and force patterns. Tactile dynamics provide interaction-aware cues to distinguish manipulation processes with similar motions but different contact states. We introduce tactile-guided optimal transport (TacOT), a framework for human-to-
robot contact-rich manipulation. TacOT leverages action–tactile dynamic time warping to identify human–
robot demonstration correspondences with consistent interaction dynamics and uses these correspondences to guide soft optimal transport
alignment in a shared policy representation space. This enables human demonstrations to provide contact-rich supervision for
robot policy learning without requiring predefined frame-level human–
robot pairing. Across four real-world dexterous manipulation tasks, TacOT improves closed-loop success rates over action-guided OT by up to 17 points on in-distribution tasks and 20 points under targeted human-to-
robot out-of-distribution transfer. Further analyses show that tactile-guided correspondence selects demonstration pairs with more consistent contact dynamics and produces latent representations that better reflect interaction-state
evolution. These results demonstrate that tactile dynamics provide an effective semantic signal for establishing reliable human-to-
robot correspondence in contact-rich dexterous manipulation.
Read the full abstract ▾
All authors Xingting Li, Yifan Han, Zijian Lin, Wei Hou, Chuqiao Lyu, Shoujie Li, Wenbo Ding
Robotics · Cross-listed from cs.CV
SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction
Yuhang Wang, Kai Luo, Yuanfan Zheng et al. (+1 more)
Comprehensive 3D scene understanding for autonomous driving requires modeling geometry, semantics, and motion. However, camera-based occupancy and scene flow prediction are sensitive to unreliable spatial and temporal aggregation, caused by semantically incompatible image features, misaligned historical observations, and incomplete voxel structures. To address this issue, we propose SelectOccFlow, a selective spatiotemporal aggregation framework that progressively refines contextual evidence across image, temporal, and voxel domains. To obtain semantically compatible image evidence, we design Semantic-Guided Sampling (SGS) to regulate feature sampling with semantic priors. Since reliable image evidence alone cannot resolve temporal inconsistency, we then present State-Conditioned Temporal Aggregation (SCTA) to selectively retrieve historical evidence according to voxel states. To further enhance the structural completeness of voxel representations, we introduce Extent-Aware Spatial Aggregation (ESA), which exploits directional structural support to refine foreground geometry. Experiments on OpenOcc demonstrate that SelectOccFlow achieves a state-of-the-art OccScore of 44.9, improving the previous best by +4.2%. It also maintains competitive occupancy performance on Occ3D-nus and improves the mean OccScore under nuScenes-C corruptions by +11.1%, demonstrating improved robustness to visual corruptions. The source code will be made publicly available at https://github.com/muchen1021/SelectOccFlow.
Read the full abstract ▾
All authors Yuhang Wang, Kai Luo, Yuanfan Zheng, Kailun Yang
Robotics
Asynchronous Tracking, Optical Communication and 3D Motion Capture using Event-based Sensors
Ziwei Wang, Angus Apps, Holly Battisson et al. (+3 more)
Inter-
robot communication is a key enabling technology in cooperative robotics applications. While wireless communication is ubiquitous, simple, and effective, it has inherent limitations: the receiving
robot cannot easily localise the source of an incoming signal, clock synchronisation is challenging, and signals are broadcast to all nearby devices rather than targeted recipients. Optical communication provides a complementary channel that is inherently directional and spatially localised, directly addressing these limitations. Event cameras are bio-inspired dynamic
vision sensors that respond to changes in image intensity with high temporal resolution, high dynamic range, and low latency, making them well-suited as receivers for high-rate optical communication in cooperative robotic systems. In this paper, we propose the Asynchronous Tracking and Optical Communication (ATOC) system, which integrates LED smart-beacon modulation, event-based detection, optical tracking, and event-data demodulation in a single pipeline. By simultaneously tracking and demodulating multiple LED smart beacons, ATOC transforms a conventional visual marker into a robust communication channel suitable for a wide range of high-impact robotics applications. We validate ATOC in a suite of laboratory studies and two 'applications': a smart-city demonstration and a 3D motion-capture system.
Read the full abstract ▾
All authors Ziwei Wang, Angus Apps, Holly Battisson, Iain Guilliard, Timothy Molloy, Robert Mahony
Robotics · Cross-listed from eess.SY
Latent Safety Filters: When a Lossy Encoder Admits a Transferable Certificate
Johannes Mootz, Zahra Nili Ahmadabadi and Reza Akhavian
Latent safety filters certify safety on a learned low-dimensional representation of the state, enabling constraints that resist analytic description. Because the encoder is lossy, a filter can report safe while the physical state is unsafe, with no detectable model error. Existing transfer conditions leave the effect of discarded safety information implicit. We ask when a lossy encoder admits a safety certificate that transfers to the physical system, and show the answer is governed by the detectability of the discarded safety-relevant dynamics. We construct a system whose latent model is exact and whose latent signals always report safe, while the physical state becomes arbitrarily unsafe. For this system no certificate exists and no monitor downstream of the encoder can detect the failure. When the discarded dynamics contract, a latent barrier certifies true safety up to two explicit margins, one for the latent-model error and one for the variation of safety across states the encoder cannot distinguish. In the linear case and under boundedness and non-degeneracy conditions, every calibrated barrier transfers with a finite margin when the safety-relevant subspace is detectable, and none does otherwise. On learned cartpole encoders, the model error does not indicate for which representations the estimated bound is non-vacuous, while the second margin does.
Read the full abstract ▾
Robotics
Real-Time Conformal-Seeded Hybrid Inverse Kinematics for Offset Redundant Manipulators
Duc Cuong Vu, Van Tung Nguyen, Duc Hai Nguyen et al. (+3 more)
This paper presents a conformal-seeded hybrid strategy for solving inverse kinematics of offset, redundant 7-DoF
robot arms of the humanoid class. Analytical inverse kinematics (AIK) provides closed-form solutions with very low computational cost. However, for offset kinematic structures, the exact closed-form solution is generally unavailable, and practical AIK must rely on an approximate or simplified kinematic model. In contrast, numerical inverse kinematics (NIK) can achieve high-precision solutions on the full kinematic model. However, its convergence is highly sensitive to initialization. To overcome these limitations, we propose a two-stage hybrid inverse kinematics framework with conformal-calibrated seed selection. First, an approximate analytical model efficiently enumerates a finite set of candidate joint solutions. Second, we rank these candidates using a lightweight learned predictor of post-refinement difficulty, wrapped by split-conformal prediction into a calibrated upper bound that serves as the selection score. The best-ranked seed is then refined using a Levenberg-Marquardt solver on the full kinematic model. The proposed method combines fast candidate generation, learned seed ranking with a calibrated difficulty bound, and accurate numerical refinement, achieving real-time performance of less than 40us and a success rate of 100% in our evaluation on reachable targets. We validate the approach through large-scale
stochastic simulation across the workspace and experimental demonstrations with motion planning on a humanoid
robot arm. Demonstration
videos are available at https://youtu.be/aeiBmw1XRbw.
Read the full abstract ▾
All authors Duc Cuong Vu, Van Tung Nguyen, Duc Hai Nguyen, Manh Cuong Nguyen, Vu Trung Tran, Minh Nhat Vu
Robotics
Grounded in Time: A Multi-Source Dataset and Benchmark for Temporal Grounding in Robotic Manipulation
Yi Wang, Yang Yang, Guangqi Xu et al. (+9 more)
Robotic manipulation often requires inferring task-relevant states from past interactions when the current observation alone is insufficient to determine the appropriate action. Despite progress in benchmarking
memory-augmented
vision-
language-action (VLA) models, application-oriented tasks requiring history-dependent semantic inference remain underrepresented. We introduce GiT (Grounded in Time), a
dataset and
benchmark for grounding manipulation decisions in past events across biolaboratory, household, and industrial scenarios. It includes real-
robot and Universal Manipulation Interface (UMI) style demonstrations covering 18 bimanual tasks, together with simulation data and a ManiSkill-based evaluation suite covering nine tasks. Fine-grained subtask annotations and annotated counterfactual task pairs, in which similar current observations require different actions depending on prior events, support policy learning and targeted evaluation of history use. Evaluations of representative end-to-end VLA models in simulation and on selected real-world tasks reveal substantial room for improvement in history-dependent manipulation. The
dataset and
benchmark are available at the project page.
Read the full abstract ▾
All authors Yi Wang, Yang Yang, Guangqi Xu, Sumin Lin, Ning Kang, Pengxiang Lu, Xiaotong Chen, Zeyu Xue, Ping Deng, Xing Liu, Chenguang Yang, Zhenyu Lu
Robotics
ROOT: Discovering Rewards for User-Specified Embodied Behaviors
Eren Sadikoglu, Aditya Taparia, Xinyuan Liu et al. (+1 more)
Reinforcement learning for embodied control remains constrained by the difficulty of reward specification. Although recent large
language model (LLM)-based methods can synthesize reward functions from natural-
language descriptions, they often fail to capture subtle behavioral properties that humans care about, such as natural gait, posture, and movement style. This limitation arises because many desired behaviors are easier to recognize visually than to encode in a reward function. We introduce Reward Optimization via Observable Trees (ROOT), a framework for discovering reward functions that align learned policies with user-specified embodied behaviors. Rather than relying solely on scalar training statistics, ROOT casts reward design as an observation-guided search over a persistent experiment tree that stores reward programs, trained policies, and rollout observations, together with behavioral insights distilled by a
video-
language model, to diagnose behavioral failures and guide subsequent reward refinements. We evaluate ROOT on seven tasks across four embodiments: simulated Hopper, HalfCheetah, Ant, Unitree Go2, and as well as the real-world Unitree Go2. ROOT produces behaviors that better align with user intent than those generated by existing LLM-based reward-generation methods, achieving up to 86.8% locomotion-completeness accuracy and improving Vid-LLM behavioral
alignment from 3.56/5 to 4.14/5, a 16.5% improvement over baselines. Human evaluations further support these results, with ROOT preferred in 51-63% of pairwise comparisons.
Read the full abstract ▾
All authors Eren Sadikoglu, Aditya Taparia, Xinyuan Liu, Ransalu Senanayake
Robotics
Attention-Based Surface Representation Learning for Robot State Prediction and Open-Ended Surface Classification
Oleg Kushnarev and Alexander Belyaev
For ground
robots operating in outdoor environments, understanding the properties of the underlying terrain is essential for ensuring reliable operation. In most perception-based studies, this problem is formulated as categorical classification with a fixed number of classes defined during training. We propose an approach that enables new surface classes to be added as trainable vectors, which can subsequently be used to address higher-level tasks. By employing a learning paradigm based on predicting the
robot's next state in time and using
attention blocks, we improved classification accuracy to 98.56% on the Belyaev-Kushnarev
dataset and 94.8% on BorealTC.
Read the full abstract ▾
Robotics
Humanoid Rickshaw Pulling: Whole-Body Locomotion under Coupled Wheeled Loads
Yangzhi Yang, Xiansheng Lin, Zhaoming Xie et al. (+1 more)
Humanoid
robots could transport payloads substantially heavier than themselves by pulling passive wheeled vehicles instead of carrying the load. This capability, however, creates a coupled locomotion problem: the
robot must maintain persistent upper-body contact while adapting to unknown, configuration-dependent forces arising from the payload, vehicle, and terrain. We present a whole-body control framework for humanoid rickshaw pulling that tracks commanded vehicle motion while preserving balance and stable grasps under uncertain load dynamics. During training, a privileged teacher exploits vehicle states, interaction forces, and load properties. Its actions and latent are distilled into a history-conditioned student that implicitly infers coupled dynamics from proprioceptive responses, followed by reinforcement-learning fine-tuning. Comparisons with No History and Only History baselines show that the resulting policy achieves accurate vehicle tracking while reducing vehicle oscillation, torso tilt, and actuation cost. Behavioral analysis shows that Unitree G1 propels the rickshaw and generates gait-synchronized whole-body reactions that stabilize its lateral and roll motions. Moreover, pulling redistributes joint effort and yields a lower
robot-normalized cost-of-transport proxy than unloaded walking over most tested load–speed conditions. On hardware, a single policy performs starting, sustained pulling, turning, and stopping with both rigid payloads and human passengers, handling a loaded rickshaw mass of up to 115~kg without load-specific retuning. These results demonstrate robust heavy-load transportation through coordinated and persistent humanoid–vehicle interaction.
Read the full abstract ▾
All authors Yangzhi Yang, Xiansheng Lin, Zhaoming Xie, Xiaobin Xiong
Robotics
Continual Humanoid Motion Learning
Zhewen He, Hao Huang, Geeta Chandra Raju Bethala et al. (+4 more)
Humanoid whole-body controllers can now track a diverse set of dynamic motions, but they are typically trained offline and then frozen, so teaching such a controller a new skill tends to erode the skills it already mastered. We study continual learning for humanoid whole-body motion, where a single controller must acquire skills from a sequential task stream without revisiting past data. We introduce Similarity-guided LoRA-PNN, a progressive neural network (PNN) policy that prevents catastrophic forgetting by construction while reusing knowledge across skills through lightweight low-rank adaptation. A two-level motion-similarity measure, built from dynamic time warping aggregated by optimal transport, decides which prior skill to build on and how much new capacity to allocate, yielding strong forward transfer and large efficiency gains. Across six sequentially learned skill categories, our similarity-guided LoRA policy attains the best forward transfer (0.125 vs. 0.079) and the highest average accuracy among all methods, while saving up to 94.5% of trainable parameters and 40.8% of training time. The resulting controller reaches 96.13% sim-to-sim transfer and is deployed on a physical Unitree G1. Our code is available at https://anonymous.4open.science/r/continual-humanoid-learning-35D3.
Read the full abstract ▾
All authors Zhewen He, Hao Huang, Geeta Chandra Raju Bethala, Chong Yu, Tao Chen, Anthony Tzes, Yi Fang
Robotics
Bi-manual Stabilization of the Cervical Spine for Safe Physical Human-Robot Interaction in Rolling Maneuvers
Elizabeth Peiros, Moira Bohley, Lucas Yager et al. (+6 more)
The neck, specifically the cervical spine, is a highly fragile and mobile part of the body with a high risk of catastrophic injury. Whether someone is injured on a battlefield, in a natural disaster, or in an athletic event, great care is always taken with the cervical spine when transporting, rolling, or moving the patient. In this work, we present an
algorithm and adaptations for bi-manual
robots to stabilize the cervical spine during rolling maneuvers. We investigate techniques to maximize grip strength within safety bounds and present a control approach for proper neck rotation. We found that with our approach, a bi-manual
robot is capable of achieving human-level performance in maintaining target rotational
alignments. The axial
alignment for the
robot with prediction was within .21 degrees of human novices.
Read the full abstract ▾
All authors Elizabeth Peiros, Moira Bohley, Lucas Yager, Derek Chen, Rebecca Fellbaum, Chris D'Ambrosia, Ananya Rao, Xiao Liang, Michael C. Yip
Robotics · Cross-listed from cs.GR
CurveCodec 2: Skeleton-agnostic animation compression with a learned entropy model
Mingyi Shi, Huancheng Lin, Xuelin Chen et al. (+1 more)
Skeletal motion is stored as every joint's transform at every frame, yet most of it is implied by the body rather than by what the motion is about. Compression is one way to ask what a motion must still say once the body is known, and a production codec must answer it for any skeleton with a stated error bound. Our earlier codec, CurveCodec, matched the mean error of ACL, the production library of modern game engines, with a learned prior over sparse anchors, but not ACL's worst case, and it counted its payload as floats rather than bits. Here we ask where the redundancy of skeletal motion lies and which part of a codec a learned model should take over. Measurements give three answers. At production precision the largest saving comes from predicting each quantized curve from its own past, the second from choosing per joint, in closed loop through the hierarchy, which samples not to code. On the gaps such an encoder leaves, a nearest-neighbour oracle over millions of training samples is no better than linear interpolation, and no learned in-betweener we tried paid for itself. What a network does learn is the distribution of the residuals the codec must send. CurveCodec 2 codes every sub-track as a curve in the log map, quantized in closed loop and thinned to rate-distortion-selected keys, with residuals
entropy-coded under a small learned model whose integer inference is bit-exact across platforms. Two contracts are verified on every decoded clip: ACL's own worst case per joint within a stated tolerance, or ACL's mean error per clip. On a held-out test side of 4,472 clips from 33
datasets, CurveCodec 2 needs 0.37x ACL's bytes at ACL's default precision of 0.01 cm under the worst-case contract and 0.22x at 0.1 cm under the mean contract, decodes on one CPU core, and transfers without retraining to a species absent from training. Project page: https://rubbly.cn/publications/curvecodec/
Read the full abstract ▾
All authors Mingyi Shi, Huancheng Lin, Xuelin Chen, Taku Komura
Robotics
Autoware in Construction: Gap Analysis and LiDAR Perception Toward Off-Road Autonomous Driving
Yu Otsuki, Teja Emmey, Sena Matsushita et al. (+2 more)
Autoware is an open-source autonomous driving software platform widely adopted by researchers and industry developers. Originally developed primarily for public-road applications, including passenger vehicles, taxis, and buses, Autoware is increasingly being extended to off-road environments such as construction and agricultural sites. Construction sites, however, differ fundamentally from public roads and challenge many assumptions underlying conventional autonomous driving systems. They are characterized by unstructured terrain, airborne dust, continuously evolving site conditions, and construction-specific objects. This paper presents lessons learned from ongoing efforts to extend an Autoware-based autonomous driving system to large dump trucks operating at construction sites. We identify gaps between public-road and off-road construction applications across the sensing, mapping, localization, perception, planning, and control modules of the Autoware stack. We then discuss potential solutions for addressing these gaps, with particular emphasis on ongoing
LiDAR-based perception pipeline development and findings from field testing. Finally, we propose a roadmap toward end-to-end autonomous driving for construction vehicles.
Read the full abstract ▾
All authors Yu Otsuki, Teja Emmey, Sena Matsushita, Akiro Harada, Takeshi Miura