Robotics
Fisheye-VLA: Decoupling Coverage and Acuity for Manipulation with a Single Fisheye Camera
Ziang Ren, Zike Yan, Raymond Zhang et al. (+2 more)
Manipulation requires both broad scene awareness and detailed local feedback, yet conventional camera rigs provide them through separate front and wrist cameras. We present Fisheye-VLA, a visual interface that brings these capabilities together using a single passive fisheye. A global view preserves the workspace, while local perspective crops direct detail toward the interaction. The key design question is where this local visual budget should go. We answer it through a controlled re-rendering study, comparing alternative crop directions on the same recorded observations. The study finds that end-effector-centered views capture most of the estimated benefit of a much larger candidate pool, motivating a compact allocation around both hands. Our interface uses calibrated end-effector projection and motion lead to track the crops, while a shared ray encoding preserves their spatial meaning as they move. Integrated with a pretrained VLA, it achieves 84% and 82% success in the two expanded tabletop regions, where some target placements extend beyond the front-camera coverage, and supports shelf and conveyor manipulation. Ablations show that local crops and their viewing directions become more important in the larger workspace regions. The results demonstrate that a single fisheye can support these manipulation tasks without physical wrist cameras.
Read the full abstract ▾
All authors Ziang Ren, Zike Yan, Raymond Zhang, Xuguo He, Zhongyu Li
Robotics
Dual Covariance Gaussian Splatting SLAM: Decoupling Rendering and Registration for Robust Real-Time Tracking
Edward Beng Wai Tan and Siew-Kei Lam
ICP-based 3D Gaussian Splatting (3DGS) SLAM tracks in real time by registering incoming frames against map Gaussians, using each primitive's covariance for both rendering and registration. These two uses place conflicting demands on one covariance. The mapper shapes it to minimize photometric error, often flattening it against surfaces, while robust registration typically benefits from measurement uncertainty. We propose a dual-covariance parameterization. Each Gaussian keeps a single mean but holds two covariances: a rendering covariance optimized by the mapper, and a tracking covariance derived from an RGB-D sensor noise model. We further use the tracking covariances as Gaussian anchors for image corners, providing constraints in directions where depth geometry is weak. We evaluate on TUM RGB-D, ScanNet, Replica, and two outdoor sequences recorded with a RealSense D435i on wheeled and handheld platforms. We achieve robust tracking performance across multiple scenes and reduced odometry drift, while tracking at $\sim$ 60 FPS.
Read the full abstract ▾
Robotics
AgriGen: Large-Scale Scene Generation Framework for Photorealistic Agricultural Robotics Simulation
Utkarsh Bajpai, Serge Tleiji, Cédric Pradalier et al. (+1 more)
Agricultural robotics is advancing rapidly, yet progress remains constrained by limited field access, lack of control over field conditions, geographic variability, and seasonal crop cycles. These factors make it difficult and costly to acquire diverse agricultural
datasets, resulting in limited evaluation and reduced system robustness. While other robotics domains have scaled learning and evaluation through high-fidelity simulation, agricultural robotics still lacks comparably capable tools. In this paper, we present a ROS-integrated framework, built on Isaac Sim, for large-scale procedural generation of agricultural environments. The framework supports photorealistic rendering, physics simulation, and domain randomization at scales relevant to robotics research, with built-in support for row crops, orchards, and vineyards and straightforward extensibility to additional crop categories. Project Page: https://baj31415.github.io/agrigen/
Read the full abstract ▾
All authors Utkarsh Bajpai, Serge Tleiji, Cédric Pradalier, Stéphanie Aravecchia
Robotics
Designing an Efficient Excavator Bucket for Lunar ISRU: A Comparative Study with Vision-Based Fill and Displacement Analysis
Abdulla Hil Kafi, Tomoki Koshi, Jorge Casir et al. (+1 more)
This paper present a spiral-cavity wheel for lunar regolith excavation and a sensor-light evaluation stack that jointly estimates fill ratio (
vision), sinkage (
vision), and specific energy from actuator logs. In benchtop tests (four revolutions at 5, 10, and 15~RPM) against two literature baselines, the proposed wheel achieved higher excavated mass and fill ratio, delivering 2.2-3.0 times higher excavation rate while reducing specific energy by 29 % relative to a bucket-drum baseline. Normalized sinkage (mm/kg) was also lower, indicating stable traction without bogging. Effort-time traces show a steady torque envelope with repeatable cut-carry-dump cycles across speeds. We provide a retention index $\eta$ that correlates with fill ratio and a DEM setup that reproduces experimental trends with low error. Results suggest spiral-cavity wheels can replace heavier multi-actuator diggers when mass, simplicity, and energy efficiency are mission drivers.
Read the full abstract ▾
All authors Abdulla Hil Kafi, Tomoki Koshi, Jorge Casir, Kenji Nagaoka
Robotics
Zephyron: Integrated Design and Analytical Evaluation of a Solar-Assisted Mobile Manipulator for Multimodal Environmental Reconnaissance and Distributed Visual Inference
Sabik Bin Sultan, Shafi Bin Sultan and Safwan Sadad
Environmental reconnaissance needs mobile platforms that carry sensors, preserve measurement context, and return interpretable evidence under limited energy and communication. We present a literature-informed engineering design for Zephyron, a four-wheel rover with a front manipulator, environmental sensors, distributed computer
vision, local recording, and a raised rear solar module. The design keeps the prototype layout but replaces unsupported numerical assumptions with an explicit component and geometry baseline. A reproducible search retrieved 5,000 records (4,858 unique) for screening, followed by targeted review of primary literature and manufacturer documentation. The baseline uses 165 mm wheels, a 12 kg mass budget, a 72 Wh battery-energy basis, and a 20 W photovoltaic module. With rolling-resistance coefficient 0.04, steady ascent of a 10 degree grade needs about 0.517 N m per wheel under equal load sharing. An illustrative 40 W motion load gives 1.44 h from 57.6 Wh usable energy, and a 25 percent driving duty gives 4.19 h without solar input; these are calculated scenarios, not measured performance. Sensor models show how integration time, calibration, temperature, and communication delay constrain interpretation, and a quality-aware stop-and-sample policy links these constraints to mission execution. Lightweight detectors, reference-based sensor learning, and executable data-integrity checks define a reproducible machine-learning evaluation pathway. The contribution is a traceable design and evaluation framework with editable 3D models, subsystem diagrams, and reproducible analytical data. Experimental validation is required before assigning payload, endurance, detection, or field-operating ratings.
Read the full abstract ▾
Robotics
The Cartesian Hand: In-Hand Manipulation with All-Linear Fingers
Boxi Xia, Bokuan Li, Ryan Shin et al. (+3 more)
Robotic manipulation has increasingly pursued human-like dexterous hands with many articulated degrees of freedom, offering rich manipulation capabilities at the cost of mechanical and control complexity. At the other extreme, parallel grippers are simple and robust, but provide little ability to manipulate an object after grasping it. Operating articulated objects such as threaded containers, manufacturing tools, and laboratory instruments often requires a second gripper, an external fixture, or coordinated arm motion. We introduce the Cartesian Hand, a 7-DoF end-effector that rethinks dexterous manipulation by combining independent grasping and relative manipulation within a single end-effector using only linear motion. Two independently actuated parallel grippers hold different parts of an object, while four translating fingertips generate relative motion between the grasped parts. Its configuration-independent fingertip kinematics allow manipulation to be composed from simple linear motion primitives. The Cartesian Hand is particularly suited to objects structured around common mechanisms such as threads, pivots, linear guides, plungers, and triggers. We demonstrate cap opening and closing, pipetting, pumping, two-handle manipulation, screwdriving, trigger actuation, and in-grasp reorientation across 35 objects spanning laboratory, manufacturing, and household settings. The same manipulation procedures transfer from a fixed-base
robot arm to a humanoid, where we demonstrate bimanual laboratory manipulation using two Cartesian Hands. These results show that versatile in-hand manipulation capability can emerge from a mechanically simple architecture when independent grasping and relative motion are designed directly into the end-effector. We will open-source all software and hardware design. Our website is https://generalroboticslab.com/cartesian_handv1.
Read the full abstract ▾
All authors Boxi Xia, Bokuan Li, Ryan Shin, Zijiang Yang, Jiaxun Liu, Boyuan Chen
Robotics
Induced Riemannian Metrics for Motion Planning with Constraints
Phone Thiha Kyaw, Thomas Cohn, Miguel Angel Rogel Garcia et al. (+1 more)
In constrained motion planning problems, task and loop-closure constraints restrict a
robot's motion to a curved, lower-dimensional submanifold of its configuration space. Planners measure path length with a metric, which sets the cost of moving in each direction. Under the Euclidean metric, this cost is the same everywhere, whereas under a general Riemannian metric, such as the kinetic-energy metric, the cost can vary with direction and configuration. Existing methods often describe the submanifold either implicitly, as a constraint level set, or explicitly, through a parameterization. The implicit representation is typically combined with the Euclidean metric of the configuration space, and the explicit representation with the parameter domain, so the path length that a planner minimizes depends on the representation. Instead, we measure path length with the induced metric, which the submanifold inherits from a Riemannian metric on the configuration space. The implicit and explicit representations yield the same induced metric, expressed in different coordinates, and hence the same geometry. This result holds for any Riemannian metric on the configuration space, not only the Euclidean one. The choice of metric is therefore independent of the choice of representation. Using this result, we extend planning under a Riemannian metric from unconstrained spaces to constraint submanifolds by applying the induced metric in both a sampling-based planner and a trajectory optimizer. For an explicit representation, the induced metric also accounts for the distortion that the parameterization introduces. In experiments on a bimanual manipulation setup with two Franka arms under end-effector task constraints, we compare the Euclidean and kinetic-energy metrics.
Read the full abstract ▾
All authors Phone Thiha Kyaw, Thomas Cohn, Miguel Angel Rogel Garcia, Jonathan Kelly
Robotics
MotionForge: A Data Generation Pipeline and Large-Scale Benchmark for Long-Horizon Manipulation of Dynamic Objects with Domain Shifts
Mohan Liu, Dengchen Mei, Haotian Xian et al. (+7 more)
Recent advances in learning-based
robot policies have demonstrated promising progress, yet they are predom- inantly evaluated in static or quasi-static environments. In dynamic manipulation, objects and scenes continuously evolve while the
robot perceives, reasons, and acts. However, recent dynamic simulation
benchmarks largely focus on short-horizon, reactive interactions with simple motion patterns and offer limited support for both systematic evaluation under domain shifts and model-agnostic real-time execution protocols. To bridge these gaps, we introduce MotionForge, the first large- scale simulation
benchmark and data-generation pipeline tailored to jointly evaluate domain shifts and long-horizon interaction in dynamic manipulation. MotionForge comprises 40 dynamic interaction tasks spanning 11 distinct motion patterns, with dedicated support for 17 long-horizon tasks. Our
benchmark introduces two key novelties: (1) a systematic evaluation protocol for assessing policy robustness under both single-factor (e.g., only backgrounds shift) and joint domain shifts (e.g., simultaneous shifts of objects, backgrounds, lighting, and speed); and (2) a decoupled, latency-aware execution protocol where the environ- ment continuously evolves independently of policy inference time. Extensive evaluations of representative general-purpose
robot policies on our
benchmark reveal substantial limitations under joint domain shifts. These findings expose a critical gap between current policy capabilities and the requirements of robust long- horizon manipulation of dynamic objects under domain shifts, establishing MotionForge as a comprehensive testbed for future research in embodied AI.
Read the full abstract ▾
All authors Mohan Liu, Dengchen Mei, Haotian Xian, Ruyang Han, Jiayi Sun, Xuanyu Chen, Haitian Zhang, Luxi Li, Kaimin Mao, Lin Wang
Robotics
MatcherCompass: A Deployment-Aware Benchmark to Guide Image Matcher Selection in the Wild
Hyunwoo Kim and Giseop Kim
Field
robots operating across time of day and sensing modalities require accurate image correspondences within onboard time and resource budgets. However, accuracy and runtime reported for individual methods on a single device provide limited guidance for choosing a matcher and its configuration on a target platform. We present MatcherCompass, a deployment-aware
benchmark for choosing local feature matchers in field robotics. Under common input and pose-evaluation procedures, we compare nine classical and learned matching pipelines across four image resolutions and supported numerical precisions. Four visual conditions cover viewpoint variation, day–night matching in visible and thermal imagery, and daytime visible–thermal matching. We evaluate pose accuracy using the area under the error–recall curve (AUC) at $5^\circ$, $10^\circ$, and $20^\circ$, and measure runtime, GPU
memory, and energy per image pair on four GPU platforms spanning workstation and onboard computers. The results show that changes in hardware, input resolution, and numerical precision can move a matcher across a runtime budget boundary, altering the feasible choices. We organize the measurements into a selection guide that returns all configurations satisfying user-specified time and resource limits, together with their accuracy under the selected visual condition. MatcherCompass provides measured evidence for choosing matching pipelines that fit a
robot's sensing conditions and computing hardware. Project page: https://matchercompass.github.io/.
Read the full abstract ▾
Robotics
SG-CPG: Severity-Gated Central Pattern Generators for Adaptive Quadruped Locomotion under Continuous Actuator Degradation
Adarsh Kumar Kosta and Kaushik Roy
An animal with a weakened limb does not necessarily switch its gait, instead it unloads the affected limb, re-coordinates the remaining limbs, and scales its response with injury severity. This graded adaptation allows locomotion to persist despite partial loss of limb strength, rather than requiring a discrete transition between healthy and failed. Inspired by this behavior, we propose SG-CPG, a central pattern generator (CPG) for quadruped locomotion under continuous actuator degradation. SG-CPG preserves a frozen healthy CPG policy and introduces two severity-driven gates: a residual gate that re-coordinates all four legs and an amplitude gate that progressively shortens the weakened leg's stride as degradation increases. We emulate progressive degradation through two mechanisms: lowering the joint torque ceiling (ceiling mechanism) and scaling its low-level controller gains (gain mechanism), representing distinct forms of actuator weakening. Our simulations on a Unitree Go2 show that SG-CPG maintains a trot gait with 100% survival across an omnidirectional command schedule under 95% joint strength loss while tracking commands within 8%. Under a lowered torque ceiling, removing either severity path, the residual's severity observation or the amplitude gate, raises clipping at the weakened joint from 4.4% to 13.6% and 26.3% of steps at an 80% loss. On a real Go2, SG-CPG survives 28 of 29 forward and turning trials with up to 93% calf torque degradation. These results show that severity-gated adaptation can extend a healthy locomotion policy to progressive actuator degradation without treating the fault as a discrete failure.
Read the full abstract ▾
Robotics
Teaching Reinforcement Learning and Humanoid Robotics to High-School Students: An Expert-Validated Curriculum Design on a Low-Cost Open Platform
Yuanzhe Dong, Jie Cao and Shuman Wang
Lower cost open source
robots and reinforcement learning (RL) simulation tools create new opportunities for precollege students to engage with contemporary robotics. However, translating a complete research workflow, spanning mechanical assembly, electrical setup, simulation, policy learning, system identification, and physical deployment, into a coherent course for novice learners remains challenging. We present an integrated robotics course framework that organizes these activities around a shared robotic artifact. The framework combines parallel disciplinary tracks, sequencing based on technical dependencies, progressive integration of simulation and hardware, layered performance checkpoints, and structures for balancing collaborative work with individual accountability. We illustrate the framework through a high school curriculum organized around a
robot project in which pairs of students assemble an open source humanoid
robot, train a walking policy in simulation, and deploy it on the physical platform. The framework was developed through an iterative design process that included formative review by five experts in robotics research, engineering, secondary STEM education, and curriculum design. Expert feedback highlighted three central design tensions: authenticity versus cognitive load, system integration versus timely visible progress, and team construction versus individual accountability. These tensions informed the final framework presented in this paper. This work offers a structured approach for adapting robotics research workflows into interdisciplinary precollege courses; future classroom studies are needed to examine implementation and student learning.
Read the full abstract ▾
Robotics
CDKF-Track: Cluster-aware Data-Driven Kalman Filtering for Cooperative 3D Multi-Object Tracking
Maria Damanaki, Nikos Piperigkos, Alexandros Gkillas et al. (+1 more)
Multi-Object Tracking (MOT) is essential for EdgeAI perception systems, where accurate object localization and reliable identification enable safe decision-making. Singleagent MOT suffers from occlusions, sensor noise, and partial scene understanding in complex real-world scenarios. While multi-
agent systems improve robustness by exploiting shared information, they introduce redundant measurements that lead to false data associations, and still struggle to capture nonlinear object dynamics. To address these challenges, we propose CDKFTrack, a Cluster-aware Data-Driven Kalman Filtering framework for Cooperative 3D MOT. The proposed method first fuses multivehicle 3D
LiDAR detections through a Graph
Laplacian-based formulation. Then, a cluster-aware redundancy reduction scheme groups spatially related detections and selects representative observations to reduce duplicate inputs to the tracker. The resulting detections are processed by a data-driven Kalman filter that learns object motion dynamics from data, reducing dependence on predefined linear motion assumptions. Furthermore, a wavelet-based temporal refinement module leverages the multiresolution decomposition property of wavelets to attenuate shortterm positional fluctuations and improve trajectory continuity. To the best of our knowledge, CDKF-Track is the first framework to jointly address detection-level fusion redundancy and learnable motion modeling in cooperative 3D MOT. Experimental results on the real-world V2V4Real
dataset indicate that CDKF-Track achieves up to 27.99% improvements in tracking accuracy over state-of-the-art multi-
agent MOT methods.
Read the full abstract ▾
All authors Maria Damanaki, Nikos Piperigkos, Alexandros Gkillas, Aris S. Lalos
Robotics
Deploying Foundation Models for Embodied Navigation
Vishnu Sashank Dorbala and Dinesh Manocha
We present and tackle two problems associated with deploying Foundation Models (FMs) on Embodied
Agents performing navigation: 1) Training bias in FMs leading to poor personalization in unseen environments, and 2) Limited FM context length hindering success, especially on long horizon tasks. Our solution for the former involves priming the FM with human-habit data mined from the scene and our solution for the latter involves active
memory management via a novel `
memory head' augmentation. We first present a taxonomy of existing literature on FM-based Embodied Navigation, and highlight these limitations. We then present our approaches, Transit-Aware Planning (TAP) and MemCtrl to address the limitations. With TAP, we present real-world results in a lab environment with a Turtlebot for personalized target finding that shows an average improvement of 18% over a non-TAP baseline. On MemCtrl, we report a 6% average improvement across various embodied tasks, with 20% on long instruction subsets, all while using nearly half the context used in the baseline model. Motivated by these result, we present our stance the deployability of FM-based embodied
agents in real-world environments, and highlight open research directions.
Read the full abstract ▾
Robotics
History-Conditioned Flow Matching for Probabilistic Dynamics of Tendon-Driven Continuum Robots
Hang Yang, Tingcong Liu, Junjie Xiong et al. (+2 more)
Deterministic dynamics modeling of tendon-driven continuum
robots remains challenging owing to uncertainties in material behavior, tendon transmission, friction, and contact. Measured joint configurations and nominal tendon commands do not fully characterize these internal mechanical factors, leaving uncertainty in the subsequent motion. We therefore develop a history-conditioned, physics-informed flow-matching framework for probabilistic dynamics prediction, using motion and actuation histories to predict the distribution of the next complete joint configuration. By recursively sampling next-step configurations under prescribed commands, the model predicts distributions of future whole-body motions. In simulation, scenario-specific models achieve five-second trajectory Energy Scores (lower is better) of 12.05 mm under internal friction variation and 9.29 mm under unobserved actuation disturbances. Relative to the conditional variational autoencoder and
diffusion baselines, Flow attains lower Energy Scores and coverage closer to the nominal level in both scenarios. Ablations support history and structural conditioning in both scenarios. On the physical
robot, predictions under two tendon-command profiles excluded from training capture the principal motion sequences, with five-second Energy Scores of 11.91 and 11.42 mm, lower than the compared baselines. The predicted-to-measured spread ratios are 1.65 and 1.22 (closer to 1 is better). These results support history-conditioned probabilistic dynamics prediction under incomplete mechanical observations.
Read the full abstract ▾
All authors Hang Yang, Tingcong Liu, Junjie Xiong, Fangju Yang, Ke Wu
Robotics
CODA: Depth-Aligned Scene Completion and Object Decomposition from a Single RGB-D Image
Dongwon Son, Junhyek Han, Yoontae Cho et al. (+5 more)
Robots operating safely in cluttered everyday environments often need to infer scene geometry from partial observations. Methods that detect objects in 2D and reconstruct them independently struggle in such scenes: a missed object is never reconstructed, a merged detection can fuse two objects, and separately reconstructed meshes may overlap or fail to touch their supporting surfaces. We introduce CODA (Complete Once, Decompose Afterward), a generative model that instead reconstructs the complete scene geometry from a single unsegmented RGB-D image, then separates the surface into the surrounding environment and movable objects. Still, generated scene geometry can drift from the observed partial point cloud. To reduce this drift, CODA uses two explicit 3D grounding mechanisms to keep reconstructed geometry consistent with observed surfaces while completing unseen regions. Experiments on HomebrewedDB and our custom cluttered-scene
dataset show more accurate reconstructions and a higher fraction of objects remaining in place under simulated
gravity than both object-first and scene-first baselines.
Read the full abstract ▾
All authors Dongwon Son, Junhyek Han, Yoontae Cho, Minseok Lee, Hong-seok Choi, Jiwook Choi, Hyungjin Kim, Beomjoon Kim
Robotics
PhyVisGen: Physically and Visually High-Fidelity Robotic Manipulation Data Generation
Yu Zheng, Qiyu Feng, Yixin Wu et al. (+6 more)
Large-scale manipulation demonstrations are essential for learning robust visuomotor policies, yet real-world data collection is expensive and difficult to scale. Simulation offers a promising alternative, but physical and visual discrepancies can limit the transferability of synthetic data, particularly for manipulation with soft grippers. We present PhyVisGen, a physically and visually high-fidelity framework for scalable robotic manipulation data generation. On the physical side, PhyVisGen introduces an arm-gripper coupling method based on the Incremental Potential Contact (IPC), enabling high-fidelity soft contact throughout complete manipulation trajectories. On the visual side, it combines real-scene reconstruction with real-time path tracing to generate visually realistic observations while preserving captured scene appearance. Quantitative evaluations demonstrate the physical and visual fidelity of PhyVisGen. Policies trained exclusively on synthetic manipulation demonstrations achieve 65-95% success across five real-
robot tasks, without real-
robot demonstration data or policy fine-tuning.
Read the full abstract ▾
All authors Yu Zheng, Qiyu Feng, Yixin Wu, Baoquan Yang, Yixuan Zhou, Bingyang Hu, Kemeng Huang, Guansheng Yang, Hesheng Wang
Robotics
Skill Sequence Planning for Collaborative Multi-Robot Construction
Xi Wang, Bo Fu, Carol C. Menassa et al. (+2 more)
Robots have significant potential to automate construction processes. However, their industry adoption remains limited, partly because of the programming effort required to adapt
robots to diverse tasks. This paper presents a skill sequence planning method that enables a heterogeneous team of multi-functional
robots to collaboratively perform construction assembly work using reusable, preprogrammed skills such as grasping, drilling, and fastening. A central controller transforms the digital representation of the building into a construction relationship graph that represents construction entities, their states, and their parent-child relationships. Based on this representation, the system selects the next construction target, generates a symbolic sequence of skills for capable members of the
robot team, and produces collision-free
geometric motion plans for skill execution. The symbolic planning problem is dynamically regenerated as the construction state changes. An interactive digital twin presents the planned skill sequence and
robot states to human co-workers for review and approval before execution. The method is evaluated through a construction assembly case study. By reducing the need to program
robots separately for each task variation, the proposed approach supports more flexible deployment of collaborative
robot teams in construction.
Read the full abstract ▾
All authors Xi Wang, Bo Fu, Carol C. Menassa, Vineet R. Kamat, Min Deng
Robotics
Contact-Stable Deformable Tissue Simulation Using Implicit Integration and Live-Pose Grasp Constraints for Laparoscopic Surgery Robot Policy Evaluation
Juahn Oh, Dongho Yee, Jinseok Lee et al. (+9 more)
Closed-loop evaluation of surgical
robots requires tissue that deforms, can be grasped and lifted, and reproduces the anatomy in which the
robot will operate. We present a simulator in which this tissue is reconstructed from a fixed-view RGB-D recording of the surgical field, composited to remove the instruments, closed into watertight volumes and tetrahedralised; the pipeline was applied unchanged to three specimens of two species (thirteen organs, 146,061 tetrahedra, no inverted elements). For one specimen, the organs are placed in a bimanual
cell in which two Franka FR3 arms operate motorised instruments through 6 mm trocars. The core contribution is the numerical and contact design that keeps this
cell stable: implicit integration, simulation meshes separate from collision meshes, numerical guards, and a grasp constraint captured at the live tissue pose. In 45 repeated grasp-lifts, a friction grasp held the tissue in 0 of 15 trials and each constraint grasp in 13 of 15; on displaced tissue, a rest-pose constraint produced one-step snaps of up to 17.8 mm, which live-pose capture eliminates. Against the recording, front-surface depth error is 1.33 to 1.41 mm, organ silhouette IoU is 0.80, and in five grasp-lifts reproduced from
video the landmark displacement RMSE is 11.8 mm against 14.2 mm for a static prediction. Biofidelity is not claimed; the environment is intended for closed-loop feasibility, safety, contact and policy screening.
Read the full abstract ▾
All authors Juahn Oh, Dongho Yee, Jinseok Lee, Jiyul Lee, Yechan Seo, Seong Jeong, Minsung Kim, Seonho Shim, Younghoon Noh, Hyuk Choi, Youngbin Kong, Hyoun-Joong Kong
Robotics
A Reconfigurable Bidirectional Cable-Driven Hip Exoskeleton with Swappable Bench/Backpack Dual-configuration Actuation
YuanLong Ji, Shuhan Xiang, Qihan Ye et al. (+1 more)
Hip exoskeletons provide an important hardware basis for lower-limb rehabilitation and locomotor assistance. Laboratory rehabilitation assessment and system development require substantial actuation and computing resources, whereas mobile assistance requires untethered portability. Integrating both capabilities within one reusable platform remains a central design challenge. This paper presents a reconfigurable bidirectional cable-driven hip exoskeleton platform that rapidly switches between bench-mounted and backpack-mounted actuation while sharing one cable-free wearable hip interface. The platform modularly adapts the actuation configuration, end-effector sensing path, and low-level control interface. Each cable-driven end-effector weighs 0.405 kg, excluding the cable and actuation unit, and integrates an encoder and a torque sensor; experiments validated bench-mounted admittance-based motion tracking capability and backpack-mounted open-loop torque tracking. Human-worn experiments with three healthy participants used myoMOTION to evaluate the platform's wearable-side hip-motion sensing capability, verified bench-to-backpack and backpack-to-bench motion-ready switching across 30 trials in $30.1\pm16.3$ s, and formed a small-scale
multimodal wearable-exoskeleton gait
dataset for sensing validation and data-driven
algorithm development, comprising 8 min bench-mounted treadmill records and 11 min backpack-mounted outdoor walking records. These results show that, by unifying the wearable structure, actuation interface, and sensing path, the proposed platform enables validation of the same hip exoskeleton in both bench-mounted and backpack-mounted configurations, providing reusable hardware for iterative development and applications across scenarios.
Read the full abstract ▾
All authors YuanLong Ji, Shuhan Xiang, Qihan Ye, Xingbang Yang
Robotics
RoboFollow: Unveiling the Instruction Following Mirage in Embodied Agents
Chang Guo, Yukun Xie, Bohan Tan et al. (+6 more)
Modern embodied
agents achieve impressive success rates, yet their actual instruction-following ability is far weaker than these numbers suggest. We trace this illusion to a structural property we term low scene
entropy: when a visual scene admits only one valid task,
language becomes redundant and a policy can score highly while barely using it. We introduce RoboFollow, a diagnostic
benchmark with three principles: (1) High Scene
Entropy: each training scene supports multiple kinematically distinct task branches, making
vision alone insufficient and forcing reliance on
language. (2) Hierarchical Diagnostic Protocol: a four-level protocol (L0–L3) progressively perturbs visual layout and semantics, probing whether equivalent instructions yield consistent behavior and distinct ones yield discriminable behavior across spatial relations, attributes, trajectory constraints, and logic. (3) Confound-Controlled Diagnosis: we simplify interaction objects, restrict actions to the trained repertoire and report stage-wise Intent and Execution scores, isolating comprehension from motor execution. Evaluation of nine VLA and WAM policies shows that strong L0 performance, where attained, does not reliably transfer to L1–L3 under our fine-tuning setup. Representative mitigations, including stronger VLM backbones, QA co-training, LangForce, and Classifier-Free Guidance, all fail to close this gap. RoboFollow exposes genuine instruction following as a critical, overlooked bottleneck. Code and
dataset are available at https://github.com/AutoLab-SAI-SJTU/RoboFollow and https://huggingface.co/
datasets/AutoLab-SJTU/robofollow-data.
Read the full abstract ▾
All authors Chang Guo, Yukun Xie, Bohan Tan, Zheng Chang, Zhaokai Yin, Qianli Ma, Yingqiao Wang, Chao Liang, Zhipeng Zhang
Robotics
DynaForge: Planning-Guided Residual Learning for Dynamic Manipulation Demonstration Generation
Yiyang Jin, Yu Zheng, Xiao He et al. (+1 more)
Dynamic object manipulation is essential for
robots operating in real-world environments, yet methods for generating high-quality demonstrations remain limited. Methods designed for static tasks do not readily transfer to dynamic settings. Among dynamic demonstration generators, planning-based methods can fail near contact, while DOMINO-style replay simplifies dynamic interactions and may limit the experience available for policy learning. We present DynaForge, a planning-guided framework that learns residual corrections for dynamic manipulation demonstration generation. DynaForge combines low-frequency global planning with high-frequency object-centric inverse kinematics across task phases, and applies a residual policy to correct actions during dynamic interaction. An implicit curriculum groups rollouts under matched conditions and selects mixed-success groups, focusing residual reinforcement learning on the evolving competence frontier. On Can and Bottle, it uses 0.73x as many optimizer steps as vanilla GRPO at the same nominal environment-step budget, with higher observed final success rates. Across nine simulation tasks, DynaForge increases mean demonstration-generation success from 41.30% of the planning prior to 78.37%. With 800 demonstrations per task, DP3 policies trained on DynaForge data achieve 49.11% mean success, compared with 7.07% for DOMINO data. On three real-world dynamic tasks, DynaForge-trained policies achieve 30-60% success, compared with 0-10% for DOMINO-trained policies, showing the ability of DynaForge for sim-to-real transfer.
Read the full abstract ▾
All authors Yiyang Jin, Yu Zheng, Xiao He, Hesheng Wang
Robotics
PAKT: Physically-Aligned Kinesthetic Teaching for Reinforcement Learning
Lars Johannsmeier and Yashraj Narang
Real-world reinforcement learning (RL) systems still struggle with the demands of contact-rich industrial manipulation, including micrometer-level precision, success rates above 99%, and human-level cycle times. Although off-policy
algorithms can improve performance by leveraging demonstrations and interventions, a key bottleneck is the lack of an intuitive interface for collecting such guidance while complying with constraints of the physical system and the policy. We propose PAKT, a framework for kinesthetic teaching in RL. As opposed to teleoperation approaches, PAKT relies on kinesthetic guidance, which is widely used in industry. However, a critical weakness of kinesthetic guidance is the possibility for the operator to move the
robot along trajectories (e.g., velocities, accelerations, jerk) that the
robot and/or policy cannot physically reproduce. Using PAKT, operators guide the
robot through admittance control, which maps human-applied forces to motion. The downstream reference generator applies the same kinematic limits used during policy execution, keeping the collected trajectories within these limits. To support this teaching interface with an appropriate execution layer, PAKT adds a high-performance control stack that maps low-frequency RL actions to high-frequency torque commands. It consists of a reference generator and subsequent impedance controller, where the reference generator preserves the tracking performance of the impedance controller while improving contact handling and producing smoother policy actions. Across the reported runs on four insertion and industrial assembly
benchmarks, including a data center compute tray, the end-to-end system reduces cycle time by 23%-48% and cumulative intervention count by 62%-86% relative to the HIL-SERL baseline. Project website: https://pakt-website.github.io/pakt-website}{https://pakt-website.github.io/pakt-website
Read the full abstract ▾
Robotics
MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligence
Haoran Wen, Wenfu Wang, Kunsong Shi et al. (+15 more)
General-purpose
robot control requires models to understand task intent, identify where to interact, capture how the scene evolves, and generate precise actions.
Vision-
language-action models provide strong semantic priors but typically do not explicitly model scene dynamics, while world-action models couple visual prediction with control without necessarily exposing the task-relevant semantic and spatial structure needed for fine-grained manipulation. We present MachEmbodied-U0 (ME-U0), a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-
Transformers architecture. Subtask prediction and affordance grounding guide joint visual-dynamics and action generation via flow matching. Visual dynamics encompass future RGB, depth, surface normals, and optical flow, providing complementary supervision for appearance, geometry, and motion. Multi-rate Rotary Position Encoding (MRPE) aligns visual dynamics with fine-grained control. We pretrain ME-U0 on approximately 4,200 hours of curated demonstrations from robotic
datasets and egocentric
datasets. Using only the supervision natively available in each downstream
benchmark, ME-U0 achieves an average score of 17.66 on the RoboDojo simulation
benchmark and average success rates of 99.0% and 82.5% on LIBERO and LIBERO-Plus, respectively. We additionally validate ME-U0 on real-world robotic manipulation tasks, demonstrating its effectiveness beyond simulation. Without corresponding downstream supervision, ME-U0 further demonstrates zero-shot subtask prediction, affordance grounding, and visual dynamics on simulated and real-world observations. Overall, ME-U0 combines competitive downstream control performance with transferable task-grounding and visual-dynamics capabilities across simulation and the real world.
Read the full abstract ▾
All authors Haoran Wen, Wenfu Wang, Kunsong Shi, Jingke Wang, Wancheng Feng, Yiren Zhang, Yueran Zhao, Xuancheng Zhang, Nanfei Ye, Xingru Chen, Zhaohong Sun, Chengmin Yang, Zikang Yu, Penghao Bi, Jia Shi, Yu Liu, Kun Zhan, Yan Xie
Robotics
From Instrument-Mounted Demonstrations to In-Vivo Execution: Learning Bimanual Laparoscopic Appendectomy Without Robot-Collected Demonstrations
Dongho Yee, Juahn Oh, Jinseok Lee et al. (+10 more)
Most minimally invasive surgery is still performed with hand-held laparoscopic instruments, and the surgeon's instrument kinematics are lost when the operation ends; only the endoscope
video is kept. This paper presents an end-to-end pipeline that captures this motion in the operating room and uses it to train a surgical
robot policy, validated on live animals. We introduce a surgical instrument-state logger that mounts on the shaft of a standard laparoscopic instrument and recovers its pose and jaw state from an inertial sensor, a time-of-flight sensor and a Hall sensor, with no external camera or tracker. A data pipeline measures the latency of every sensor channel against a
robot ground truth and aligns the channels before forming observation-action pairs. On these demonstrations we train a
diffusion policy with a fine-tuned DINOv3 backbone, selecting its design by closed-loop rollouts in a physics simulator reconstructed from depth maps of an ex-vivo rabbit appendix. The policy is then retrained on 849 in-vivo demonstrations from four live rabbits and deployed on four additional live rabbits with electrosurgery armed. With the surgeon selecting the surgical phase, the policy completed the appendectomy in three of the four animals. The results show that demonstrations recorded from a surgeon's own instruments are sufficient to train, select and deploy a bimanual surgical policy in vivo. The
robot serves only as the timing reference for sensor calibration and as the executor, and collects no demonstrations. Both demonstration corpora are released to support future surgical
robot learning research.
Read the full abstract ▾
All authors Dongho Yee, Juahn Oh, Jinseok Lee, Jiyul Lee, Yechan Seo, Seong Jeong, Minsung Kim, Seonho Shim, Younghoon Noh, Hyuk Choi, Youngbin Kong, Kyu Eun Lee, Hyoun-Joong Kong
Robotics
Relative Contact Velocity-Controlled Hand-Object Mechanism for Dexterous Tool Manipulation
Sunyu Wang, Jean Oh and Nancy S. Pollard
This work investigates how to enable general multi-finger robotic hands to perform the complete tool manipulation process, which entails picking up a tool, loading it into a suitable pose, and then wielding it. Inspired by human tool manipulation and mechanical design principles, we model the hand and the tool as a unified hand-object mechanism (HOM) composed of sub-assemblies. Specifically, we define a HOM as consisting of the hand, the object, and the generalized contact frames, allowing the HOM's motions to be expressed with the same set of Cartesian-space relative contact velocities, irrespective of the hand's kinematics and geometry. Then, we define a HOM's sub-assemblies as relative contact velocity and contact force constraints between fingers. Building on these definitions, we developed a lightweight and physically interpretable motion planning and contact estimation framework using least squares and a complementary filter. We evaluated our framework in simulation by teleoperating five different robotic hands. The results show that our framework enabled all five hands to execute the complete tool manipulation process, achieving dexterous behaviors even from identical, simple reference trajectories. Furthermore, the results showcase our framework's adaptability to different hands, tools, and tasks, enabled by its kinematic and
geometric foundation. Please visit website for
videos: \href{https://sunyuw.github.io/hom-sim/}{https://sunyuw.github.io/hom-sim/}
Read the full abstract ▾
Robotics
A Deployable Four-Finger Payload for Teleoperated Free-Flying Manipulation with Astrobee
William Su, Jordan Kam, Yunosuke Nakamura et al. (+3 more)
This article presents a bimanual teleoperation pipeline and conceptual design of a deployable four-finger payload for intra-vehicular free-flyers. Future habitats in low-Earth orbit (LEO) will require systems to perform mundane tasks like cargo handling and maintenance during crewed and uncrewed periods. The gripper payload provides 17 manipulation degrees-of-freedom (DoF) through four independently actuated fingers on a linear rail system. To control it, a virtual reality (VR) device interface maps the human ground operator's hand motions to the finger pairs, their separation to the rail, and common wrist motion to Astrobee translation. We present the preliminary results of teleoperating Astrobee in a custom zero-
gravity MuJoCo-based International Space Station (ISS) simulator through ten repeated trials of transporting a rigid ISS Cargo Transfer Bag (CTB). We measure task success, continuous contact retention, completion time, and cargo motion.
Read the full abstract ▾
All authors William Su, Jordan Kam, Yunosuke Nakamura, Yixiao Wang, Jianshu Zhou, Masayoshi Tomizuka
Robotics
CableVLA: Simulation-Privileged Global-Local Representation Learning for Cable Routing
Zhifei Teng, Bo Feng, Xiang Zou et al. (+4 more)
Cable routing requires coordinated control of global cable
topology and changing local contacts. We present CableVLA, an end-to-end
multimodal vision-
language-action framework that converts simulation-privileged supervision into deployable cable-
topology and tactile representations. TopoHead distills node-level physics and current and future cable-
topology information into causal visual context for the action expert. TacSense uses complementary frame and taxel branches to learn contact dynamics from resistive arrays, with simulator-derived kinematics and contact events providing supervision beyond the measured force map. A contact gate activates force-tactile residuals that refine the next 8 arm-and-gripper actions of a frozen
topology-conditioned policy. Across 345 MuJoCo evaluations, CableVLA improves success from 62.6% for the $\pi_{0.5}$-V visual baseline to 84.9%. TacSense achieves pronounced gains in slip-transition recognition over a CNN-LSTM baseline with a similar parameter count, and this advantage persists under frozen-encoder probes.
Topology prediction and 57-task tactile evaluations assess representation quality, while policy adaptation studies evaluate downstream control performance. Cross-simulator and real-
robot comparisons further examine zero-shot policy transfer under changes in dynamics and sensing.
Read the full abstract ▾
All authors Zhifei Teng, Bo Feng, Xiang Zou, Jinpeng Xiao, Min Li, Zhouping Yin, Yiqun Li
Robotics
Recording Hand-Held Laparoscopic Instrument Motion in the Operating Room: Magnetometer-Free Fusion of Inertial, Range and Visual Sensing
Jiyul Lee, Dongho Yee, Juahn Oh et al. (+9 more)
Most minimally invasive procedures are still performed with hand-held laparoscopic instruments, yet only the endoscopic
video is retained; the instrument motion that expresses surgical skill, and that could support skill assessment and
robot learning, is lost. Pose from
video alone remains millimeters to centimeters off, and an instrument-mounted inertial measurement unit (IMU) cannot rely on its magnetometer, whose field changed with tool pose and between sessions in our measurements. We present a surgical instrument-state logger that clips onto a conventional instrument without modifying the part that enters the patient and fuses a six-axis IMU and a time-of-flight (ToF) rangefinder with a markerless camera in an error-state Kalman filter under the remote center of motion (RCM) of the trocar. Heading comes from the shaft silhouette, segmented by a U-Net, in place of the magnetometer: the rotation-angle error is 0.200{\deg}, against 3.58{\deg} from the accelerometer and magnetometer alone. Against a Franka Research 3 manipulator, and without
alignment to it, the displacement error over 300 translation trials was 1.21mm RMS and the relative-rotation error over 180 rotation trials 0.34{\deg} RMS. On continuous trajectories, tracked and displayed in real time, the absolute tip error was 1.22mm (programmed) and 3.04mm (teleoperated) after post-hoc tuning of three filter parameters, and the full fusion beat every sensor subset. Because the
estimator uses no
magnetic measurement, its accuracy does not rely on an undisturbed field. The same clip-on device could thus record metric tip trajectories during routine hand-held laparoscopy, while displaying the insertion depth and attitude that are hidden once the instrument is inside the patient.
Read the full abstract ▾
All authors Jiyul Lee, Dongho Yee, Juahn Oh, Jinseok Lee, Yechan Seo, Seong Jeong, Minsung Kim, Seonho Shim, Younghoon Noh, Hyuk Choi, Youngbin Kong, Hyoun-Joong Kong
Robotics
IndustrialVLA-Bench: A Traceable Multi-Axis Evaluation of Open Robot Policy Models
Yiqi Wang, Zhifeng Rao, Jiaqi Zhang et al. (+7 more)
Open
robot policies increasingly follow two paradigms:
vision-
language-action models (VLAs) directly map observations and instructions to actions, whereas world-action models (WAMs) incorporate learned
video or world dynamics into policy learning or action generation. Although both target the same manipulation tasks and represent alternative design choices, they are commonly reported under different evaluation protocols, leaving their capability, robustness,
language sensitivity, and deployment-cost trade-offs unclear. We present IndustrialVLA-Bench, an evidence-aware evaluation of six released VLA and WAM systems under a unified reporting schema. It separately evaluates clean capability on LIBERO, non-
language robustness on LIBERO-Plus, instruction sensitivity on LIBERO-Para, and observed execution cost. Reported task scores aggregate three complete evaluations with distinct random seeds under a fixed checkpoint and inference configuration. Across all six systems, clean LIBERO averages differ by only 1.58 points, whereas robustness and paraphrase summaries span 14.62 and 31.08 points. Restricting every comparison to the three protocol-faithful systems preserves the effect (1.36, 14.62 and 23.10 points), so the diagnostic separation reported here does not depend on the weaker evidence tiers. We additionally report observed inference latency, peak
memory, runtime mode, and an evidence status for every system. Protocol-faithful, near-reproduction, and pending-verification entries remain visibly separated; only protocol-faithful entries support strict comparisons. Rather than claiming universal superiority of either paradigm, IndustrialVLA-Bench provides traceable evidence for comparing released
robot policies on shared practical criteria. Code and evaluation records are available at https://github.com/xiaoqi-7/IndustrialVLA-Bench.
Read the full abstract ▾
All authors Yiqi Wang, Zhifeng Rao, Jiaqi Zhang, Xiaoyang Li, Zhangkai Wu, Yiqun Duan, Mingkai Zheng, Fei Wang, Shan You, Taotao Cai
Robotics
HABILIS Brain 0: Geometry-Change Supervision for Vision-Language-Action and Residual Flow Recovery
Jinu Pahk, Jesoon Kang, Taegeon Park et al. (+4 more)
Vision-
language-action policies benefit from
geometric supervision, but current-frame geometry alone does not explicitly describe the changes associated with manipulation. This design is motivated by the goal of learning an embodiment-agnostic visual interface that can be pretrained across
robot and egocentric
video before
robot-specific action
alignment. We introduce Geometry-Change VLA (GC-VLA), which learns to predict multiview future-current geometry-change
tokens from current observations. Offline frame pairs define a nominal 0.5-second prediction horizon; future observations are used only to construct training targets. Stage 1 trains a geometry-change
vision-
language model (GC-VLM). Stage 2 introduces a continuous ActionExpert and aligns it with
robot actions while stopping action-flow gradients at the VLM interface. Stage 3 enables these gradients to update the trainable VLM components jointly with the ActionExpert. Stage 4 freezes GC-VLA and applies Geometry-Conditioned Residual Flow (GCRF), using a binary intervention router and a single bounded residual velocity policy learned from closed-loop feedback. GC-VLA achieves 95.20% success on LIBERO, and GC-VLA with GCRF achieves 99.55%. Inference uses current observations and the learned GC representation without executing the offline target encoders.
Read the full abstract ▾
All authors Jinu Pahk, Jesoon Kang, Taegeon Park, Jisu An, Soo Min Kimm, Jaejoon Kim, Byoung-Tak Zhang