Peer Revenue Past citations are no guarantee of future returns

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Robotics

cs.RO Watch Robotics

Sunday 8 February

Robotics

Global Symmetry and Orthogonal Transformations from Geometrical Moment $n$-tuples

Omar Tahri

Detecting symmetry is crucial for effective object grasping for several reasons. Recognizing symmetrical features or axes within an object helps in developing efficient grasp strategies, as grasping along these axes typically results in a more stable and balanced grip, thereby facilitating successful manipulation. This paper employs geometrical moments to identify symmetries and estimate orthogonal transformations, including rotations and mirror transformations, for objects centered at the frame origin. It provides distinctive metrics for detecting symmetries and estimating orthogonal transformations, encompassing rotations, reflections, and their combinations. A comprehensive methodology is developed to obtain these functions in n-dimensional space, specifically moment \( n \)-tuples. Extensive validation tests are conducted on both 2D and 3D objects to ensure the robustness and reliability of the proposed approach. The proposed method is also compared to state-of-the-art work using iterative optimization for detecting multiple planes of symmetry. The results indicate that combining our method with the iterative one yields satisfactory outcomes in terms of the number of symmetry planes detected and computation time.
Read the full abstract

Saturday 7 February

Robotics · Cross-listed from cs.CV

Vision and Language: Novel Representations and Artificial intelligence for Driving Scene Safety Assessment and Autonomous Vehicle Planning

Ross Greer, Maitrayee Keskar, Angel Martinez-Sanchez et al. (+3 more)

Vision-language models (VLMs) have recently emerged as powerful representation learning systems that align visual observations with natural language concepts, offering new opportunities for semantic reasoning in safety-critical autonomous driving. This paper investigates how vision-language representations support driving scene safety assessment and decision-making when integrated into perception, prediction, and planning pipelines. We study three complementary system-level use cases. First, we introduce a lightweight, category-agnostic hazard screening approach leveraging CLIP-based image-text similarity to produce a low-latency semantic hazard signal. This enables robust detection of diverse and out-of-distribution road hazards without explicit object detection or visual question answering. Second, we examine the integration of scene-level vision-language embeddings into a transformer-based trajectory planning framework using the Waymo Open Dataset. Our results show that naively conditioning planners on global embeddings does not improve trajectory accuracy, highlighting the importance of representation-task alignment and motivating the development of task-informed extraction methods for safety-critical planning. Third, we investigate natural language as an explicit behavioral constraint on motion planning using the doScenes dataset. In this setting, passenger-style instructions grounded in visual scene elements suppress rare but severe planning failures and improve safety-aligned behavior in ambiguous scenarios. Taken together, these findings demonstrate that vision-language representations hold significant promise for autonomous driving safety when used to express semantic risk, intent, and behavioral constraints. Realizing this potential is fundamentally an engineering problem requiring careful system design and structured grounding rather than direct feature injection.
Read the full abstract

All authors Ross Greer, Maitrayee Keskar, Angel Martinez-Sanchez, Parthib Roy, Shashank Shriram, Mohan Trivedi

Robotics

Affine Transformable Unmanned Ground Vehicle

Aron Mathias, Mohammad Ghufran, Jack Hughes et al. (+1 more)

This paper develops the proof of concept for a novel affine transformable unmanned ground vehicle (ATUGV) with the capability of safe and aggressive deformation while carrying multiple payloads. The ATUGV is a multi-body system with mobile robots that can be used to power the ATUGV morphable motion, powered cells to enclose the mobile robots, unpowered cells to contain payloads, and a deformable structure to integrate cells through bars and joints. The objective is that all powered and unpowered cells motion can safely track a desired affine transformation, where an affine transformation can be decomposed into translation, rigid body rotation, and deformation. To this end, the paper first uses a deep neural network to structure cell interconnection in such a way that every cell can freely move over the deformation plane, and the entire structure can reconfigurably deform to track a desired affine transformation. Then, the mobile robots, contained by the powered cells and stepper motors, regulating the connections of the powered and unpowered cells, design the proper controls so that all cells safely track the desired affine transformation. The functionality of the proposed ATUGV is validated through hardware experimentation and simulation.
Read the full abstract

All authors Aron Mathias, Mohammad Ghufran, Jack Hughes, Hossein Rastgoftar

Robotics · Cross-listed from cs.CV

Looking and Listening Inside and Outside: Multimodal Artificial Intelligence Systems for Driver Safety Assessment and Intelligent Vehicle Decision-Making

Ross Greer, Laura Fleig, Maitrayee Keskar et al. (+9 more)

The looking-in-looking-out (LILO) framework has enabled intelligent vehicle applications that understand both the outside scene and the driver state to improve safety outcomes, with examples in smart airbag deployment, takeover time prediction in autonomous control transitions, and driver attention monitoring. In this research, we propose an augmentation to this framework, making a case for the audio modality as an additional source of information to understand the driver, and in the evolving autonomy landscape, also the passengers and those outside the vehicle. We expand LILO by incorporating audio signals, forming the looking-and-listening inside-and-outside (L-LIO) framework to enhance driver state assessment and environment understanding through multimodal sensor fusion. We evaluate three example cases where audio enhances vehicle safety: supervised learning on driver speech audio to classify potential impairment states (e.g., intoxication), collection and analysis of passenger natural language instructions (e.g., "turn after that red building") to motivate how spoken language can interface with planning systems through audio-aligned instruction data, and limitations of vision-only systems where audio may disambiguate the guidance and gestures of external agents. Datasets include custom-collected in-vehicle and external audio samples in real-world environments. Pilot findings show that audio yields safety-relevant insights, particularly in nuanced or context-rich scenarios where sound is critical to safe decision-making or visual signals alone are insufficient. Challenges include ambient noise interference, privacy considerations, and robustness across human subjects, motivating further work on reliability in dynamic real-world contexts. L-LIO augments driver and scene understanding through multimodal fusion of audio and visual sensing, offering new paths for safety intervention.
Read the full abstract

All authors Ross Greer, Laura Fleig, Maitrayee Keskar, Erika Maquiling, Giovanni Tapia Lopez, Angel Martinez-Sanchez, Parthib Roy, Jake Rattigan, Mira Sur, Alejandra Vidrio, Thomas Marcotte, Mohan Trivedi

Robotics

LCLA: Language-Conditioned Latent Alignment for Vision-Language Navigation

Nitesh Subedi, Adam Haroon, Samuel Tetteh et al. (+3 more)

We propose LCLA (Language-Conditioned Latent Alignment), a framework for vision-language navigation that learns modular perception-action interfaces by aligning sensory observations to a latent representation of an expert policy. The expert is first trained with privileged state information, inducing a latent space sufficient for control, after which its latent interface and action head are frozen. A lightweight adapter is then trained to map raw visual-language observations, via a frozen vision-language model, into the expert's latent space, reducing the problem of visuomotor learning to supervised latent alignment rather than end-to-end policy optimization. This decoupling enforces a stable contract between perception and control, enabling expert behavior to be reused across sensing modalities and environmental variations. We instantiate LCLA and evaluate it on a vision-language indoor navigation task, where aligned latent spaces yield strong in-distribution performance and robust zero-shot generalization to unseen environments, lighting conditions, and viewpoints while remaining lightweight at inference time.
Read the full abstract

All authors Nitesh Subedi, Adam Haroon, Samuel Tetteh, Prajwal Koirala, Cody Fleming, Soumik Sarkar

Robotics

"Meet My Sidekick!": Effects of Separate Identities and Control of a Single Robot in HRI

Drake Moore, Arushi Aggarwal, Emily Taylor et al. (+3 more)

The presentation of a robot's capability and identity directly influences a human collaborator's perception and implicit trust in the robot. Unlike humans, a physical robot can simultaneously present different identities and have them reside and control different parts of the robot. This paper presents a novel study that investigates how users perceive a robot where different robot control domains (head and gripper) are presented as independent robots. We conducted a mixed design study where participants experienced one of three presentations: a single robot, two agents with shared full control (co-embodiment), or two agents with split control across robot control domains (split-embodiment). Participants underwent three distinct tasks – a mundane data entry task where the robot provides motivational support, an individual sorting task with isolated robot failures, and a collaborative arrangement task where the robot causes a failure that directly affects the human participant. Participants perceived the robot as residing in the different control domains and were able to associate robot failure with different identities. This work signals how future robots can leverage different embodiment configurations to obtain the benefit of multiple robots within a single body.
Read the full abstract

All authors Drake Moore, Arushi Aggarwal, Emily Taylor, Sarah Zhang, Taskin Padir, Xiang Zhi Tan

Robotics

Differentiate-and-Inject: Enhancing VLAs via Functional Differentiation Induced by In-Parameter Structural Reasoning

Jingyi Hou, Leyu Zhou, Chenchen Jing et al. (+3 more)

As robots are expected to perform increasingly diverse tasks, they must understand not only low-level actions but also the higher-level structure that determines how a task should unfold. Existing vision-language-action (VLA) models struggle with this form of task-level reasoning. They either depend on prompt-based in-context decomposition, which is unstable and sensitive to linguistic variations, or end-to-end long-horizon training, which requires large-scale demonstrations and entangles task-level reasoning with low-level control. We present in-parameter structured task reasoning (iSTAR), a framework for enhancing VLA models via functional differentiation induced by in-parameter structural reasoning. Instead of treating VLAs as monolithic policies, iSTAR embeds task-level semantic structure directly into model parameters, enabling differentiated task-level inference without external planners or handcrafted prompt inputs. This injected structure takes the form of implicit dynamic scene-graph knowledge that captures object relations, subtask semantics, and task-level dependencies in parameter space. Across diverse manipulation benchmarks, iSTAR achieves more reliable task decompositions and higher success rates than both in-context and end-to-end VLA baselines, demonstrating the effectiveness of parameter-space structural reasoning for functional differentiation and improved generalization across task variations.
Read the full abstract

All authors Jingyi Hou, Leyu Zhou, Chenchen Jing, Jinghan Yang, Xinbo Yu, Wei He

Robotics

VividFace: Real-Time and Realistic Facial Expression Shadowing for Humanoid Robots

Peizhen Li, Longbing Cao, Xiao-Ming Wu et al. (+1 more)

Humanoid facial expression shadowing enables robots to realistically imitate human facial expressions in real time, which is critical for lifelike, facially expressive humanoid robots and affective human-robot interaction. Existing progress in humanoid facial expression imitation remains limited, often failing to achieve either real-time performance or realistic expressiveness due to offline video-based inference designs and insufficient ability to capture and transfer subtle expression details. To address these limitations, we present VividFace, a real-time and realistic facial expression shadowing system for humanoid robots. An optimized imitation framework X2CNet++ enhances expressiveness by fine-tuning the human-to-humanoid facial motion transfer module and introducing a feature-adaptation training strategy for better alignment across different image sources. Real-time shadowing is further enabled by a video-stream-compatible inference pipeline and a streamlined workflow based on asynchronous I/O for efficient communication across devices. VividFace produces vivid humanoid faces by mimicking human facial expressions within 0.05 seconds, while generalizing across diverse facial configurations. Extensive real-world demonstrations validate its practical utility. Videos are available at: https://lipzh5.github.io/VividFace/.
Read the full abstract

All authors Peizhen Li, Longbing Cao, Xiao-Ming Wu, Yang Zhang

Robotics

TextOp: Real-time Interactive Text-Driven Humanoid Robot Motion Generation and Control

Weiji Xie, Jiakun Zheng, Jinrui Han et al. (+4 more)

Recent advances in humanoid whole-body motion tracking have enabled the execution of diverse and highly coordinated motions on real hardware. However, existing controllers are commonly driven either by predefined motion trajectories, which offer limited flexibility when user intent changes, or by continuous human teleoperation, which requires constant human involvement and limits autonomy. This work addresses the problem of how to drive a universal humanoid controller in a real-time and interactive manner. We present TextOp, a real-time text-driven humanoid motion generation and control framework that supports streaming language commands and on-the-fly instruction modification during execution. TextOp adopts a two-level architecture in which a high-level autoregressive motion diffusion model continuously generates short-horizon kinematic trajectories conditioned on the current text input, while a low-level motion tracking policy executes these trajectories on a physical humanoid robot. By bridging interactive motion generation with robust whole-body control, TextOp unlocks free-form intent expression and enables smooth transitions across multiple challenging behaviors such as dancing and jumping, within a single continuous motion execution. Extensive real-robot experiments and offline evaluations demonstrate instant responsiveness, smooth whole-body motion, and precise control. The project page and the open-source code are available at https://text-op.github.io/
Read the full abstract

All authors Weiji Xie, Jiakun Zheng, Jinrui Han, Jiyuan Shi, Weinan Zhang, Chenjia Bai, Xuelong Li

Robotics

Bridging Speech, Emotion, and Motion: a VLM-based Multimodal Edge-deployable Framework for Humanoid Robots

Songhua Yang, Xuetao Li, Xuanye Fei et al. (+2 more)

Effective human-robot interaction requires emotionally rich multimodal expressions, yet most humanoid robots lack coordinated speech, facial expressions, and gestures. Meanwhile, real-world deployment demands on-device solutions that can operate autonomously without continuous cloud connectivity. To bridging \underline{S}peech, \underline{E}motion, and \underline{M}otion, we present \textit{SeM$^2$}, a Vision Language Model-based framework that orchestrates emotionally coherent multimodal interactions through three key components: a multimodal perception module capturing user contextual cues, a Chain-of-Thought reasoning for response planning, and a novel Semantic-Sequence Aligning Mechanism (SSAM) that ensures precise temporal coordination between verbal content and physical expressions. We implement both cloud-based and \underline{e}dge-deployed versions (\textit{SeM$^2_e$}), with the latter knowledge distilled to operate efficiently on edge hardware while maintaining 95% of the relative performance. Comprehensive evaluations demonstrate that our approach significantly outperforms unimodal baselines in naturalness, emotional clarity, and modal coherence, advancing socially expressive humanoid robotics for diverse real-world environments.
Read the full abstract

All authors Songhua Yang, Xuetao Li, Xuanye Fei, Mengde Li, Miao Li

Robotics

Going with the Flow: Koopman Behavioral Models as Pseudo Planners for Visuo-Motor Dexterity

Yunhai Han, Jiaqi Fu, Linhao Bai et al. (+7 more)

Contemporary visuo-motor dexterity models often rely on expressive policy classes with diffusion and transformer backbones to achieve strong performance. However, these architectures require significant data and computational resources, and remain far from reliable, particularly for multi-fingered dexterity. Importantly, they model skills as reactive mappings and rely on fixed-horizon action chunking, creating a rigid trade-off between temporal coherence and reactivity. To address these issues, we first introduce Unified Behavioral Models (UBMs), a framework to represent dexterous skills as coupled dynamical systems that capture how visual features of the environment (visual flow) and proprioceptive states of the robot (action flow) co-evolve. As such, UBMs ensure temporal coherence by construction rather than heuristic averaging. Unlike world models that attempt to predict the impact of arbitrary robot actions on the environment, UBMs target behavioral dynamics that encode how demonstrated robot behavior is related to desired impacts on the environment. A UBM can be viewed as a pseudo planner: given an initial condition, it computes the desired robot behavior over the entire skill horizon, while simultaneously “imagining" the resulting flow of visual features. To operationalize UBMs, we propose Koopman-UBM, a first instantiation of UBMs as a structured latent linear system. K-UBM is computationally efficient, enabling reactivity and adaptation via an online replanning strategy: the model acts as its own runtime monitor, automatically triggering replanning when predicted and observed visual flow diverge beyond a threshold. Across seven simulated tasks and four real-world tasks, our approach matches or exceeds the performance of state-of-the-art baselines, while offering considerably faster inference, smooth execution, robustness to occlusions, and flexible replanning.
Read the full abstract

All authors Yunhai Han, Jiaqi Fu, Linhao Bai, Ziyu Xiao, Zhaodong Yang, Yogita Choudhary, Krishna Jha, Chuizheng Kong, Shreyas Kousik, Harish Ravichandar

Robotics · Cross-listed from cs.HC

Haptically Experienced Animacy Facilitates Emotion Regulation: A Theory-Driven Investigation

Preeti Vyas, Bereket Guta, Tim G. Zhou et al. (+3 more)

Emotion regulation (ER) is essential to mental well-being but often difficult to access, especially in high-intensity moments or for individuals with clinical vulnerabilities. While existing technology-based ER tools offer value, they typically rely on self-reflection (e.g., emotion tracking, journaling) or co-regulation through verbal modalities (reminders, text-based conversational tools), which may not be accessible or effective when most needed. The biological role of the touch modality makes it an intriguing alternate pathway, but empirical evidence is limited and under-theorized. Building on our prior theoretical framework describing how a comforting haptic co-regulating adjunct (CHORA) can support ER, we developed a zoomorphic robot CHORA with looped biomimetic breathing and heartbeat behaviors. We evaluated its effects in a mixed-methods in-lab study (N=30), providing physiological, self-report, custom questionnaire, and retrospective interview data. Our findings demonstrate the regulatory effects of haptically experienced animacy, corroborate prior work, and validate CHORA's {theoretically grounded} potential to facilitate four ER strategies.
Read the full abstract

All authors Preeti Vyas, Bereket Guta, Tim G. Zhou, Noor Naila Himam, Andero Uusberg, Karon E. MacLean

Robotics

Trace-Focused Diffusion Policy for Multi-Modal Action Disambiguation in Long-Horizon Robotic Manipulation

Yuxuan Hu, Xiangyu Chen, Chuhao Zhou et al. (+4 more)

Generative model-based policies have shown strong performance in imitation-based robotic manipulation by learning action distributions from demonstrations. However, in long-horizon tasks, visually similar observations often recur across execution stages while requiring distinct actions, which leads to ambiguous predictions when policies are conditioned only on instantaneous observations, termed multi-modal action ambiguity (MA2). To address this challenge, we propose the Trace-Focused Diffusion Policy (TF-DP), a simple yet effective diffusion-based framework that explicitly conditions action generation on the robot's execution history. TF-DP represents historical motion as an explicit execution trace and projects it into the visual observation space, providing stage-aware context when current observations alone are insufficient. In addition, the induced trace-focused field emphasizes task-relevant regions associated with historical motion, improving robustness to background visual disturbances. We evaluate TF-DP on real-world robotic manipulation tasks exhibiting pronounced multi-modal action ambiguity and visually cluttered conditions. Experimental results show that TF-DP improves temporal consistency and robustness, outperforming the vanilla diffusion policy by 80.56 percent on tasks with multi-modal action ambiguity and by 86.11 percent under visual disturbances, while maintaining inference efficiency with only a 6.4 percent runtime increase. These results demonstrate that execution-trace conditioning offers a scalable and principled approach for robust long-horizon robotic manipulation within a single policy.
Read the full abstract

All authors Yuxuan Hu, Xiangyu Chen, Chuhao Zhou, Yuxi Liu, Gen Li, Jindou Jia, Jianfei Yang

Robotics

UEREBot: Learning Safe Quadrupedal Locomotion under Unstructured Environments and High-Speed Dynamic Obstacles

Zihao Xu, Runyu Lei, Zihao Li et al. (+3 more)

Quadruped robots are increasingly deployed in unstructured environments. Safe locomotion in these settings requires long-horizon goal progress, passability over uneven terrain and static constraints, and collision avoidance against high-speed dynamic obstacles. A single system cannot fully satisfy all three objectives simultaneously: planning-based decisions can be too slow, while purely reactive decisions can sacrifice goal progress and passability. To resolve this conflict, we propose UEREBot (Unstructured-Environment Reflexive Evasion Robot), a hierarchical framework that separates slow planning from instantaneous reflexive evasion and coordinates them during execution. UEREBot formulates the task as a constrained optimal control problem blueprint. It adopts a spatial–temporal planner that provides reference guidance toward the goal and threat signals. It then uses a threat-aware handoff to fuse navigation and reflex actions into a nominal command, and a control barrier function shield as a final execution safeguard. We evaluate UEREBot in Isaac Lab simulation and deploy it on a Unitree Go2 quadruped equipped with onboard perception. Across diverse environments with complex static structure and high-speed dynamic obstacles, UEREBot achieves higher avoidance success and more stable locomotion while maintaining goal progress than representative baselines, demonstrating improved safety–progress trade-offs.
Read the full abstract

All authors Zihao Xu, Runyu Lei, Zihao Li, Boxi Lin, Ce Hao, Jin Song Dong

Robotics · Cross-listed from cs.CV

Seeing Roads Through Words: A Language-Guided Framework for RGB-T Driving Scene Segmentation

Ruturaj Reddy, Hrishav Bakul Barua, Junn Yong Loo et al. (+2 more)

Robust semantic segmentation of road scenes under adverse illumination, lighting, and shadow conditions remain a core challenge for autonomous driving applications. RGB-Thermal fusion is a standard approach, yet existing methods apply static fusion strategies uniformly across all conditions, allowing modality-specific noise to propagate throughout the network. Hence, we propose CLARITY that dynamically adapts its fusion strategy to the detected scene condition. Guided by vision-language model (VLM) priors, the network learns to modulate each modality's contribution based on the illumination state while leveraging object embeddings for segmentation, rather than applying a fixed fusion policy. We further introduce two mechanisms - one which preserves valid dark-object semantics that prior noise-suppression methods incorrectly discard, and a hierarchical decoder that enforces structural consistency across scales to sharpen boundaries on thin objects. Experiments on the MFNet dataset demonstrate that CLARITY establishes a new state-of-the-art (SOTA), achieving 62.3% mIoU and 77.5% mAcc.
Read the full abstract

All authors Ruturaj Reddy, Hrishav Bakul Barua, Junn Yong Loo, Thanh Thi Nguyen, Ganesh Krishnasamy

Robotics · Cross-listed from cs.LG

Scalable Dexterous Robot Learning with AR-based Remote Human-Robot Interactions

Yicheng Yang, Ruijiao Li, Lifeng Wang et al. (+7 more)

This paper focuses on the scalable robot learning for manipulation in the dexterous robot arm-hand systems, where the remote human-robot interactions via augmented reality (AR) are established to collect the expert demonstration data for improving efficiency. In such a system, we present a novel method to address the general manipulation task problem. Specifically, the proposed method consists of two phases: i) In the first phase for pretraining, the policy is created in a behavior cloning (BC) manner, through leveraging the learning data from our AR-based remote human-robot interaction system; ii) In the second phase, a contrastive learning empowered reinforcement learning (RL) method is developed to obtain more efficient and robust policy than the BC, and thus a projection head is designed to accelerate the learning progress. An event-driven augmented reward is adopted for enhancing the safety. To validate the proposed method, both the physics simulations via PyBullet and real-world experiments are carried out. The results demonstrate that compared to the baselines, our method not only significantly speeds up the training process, but also achieves much better performance in terms of the success rate for fulfilling the manipulation tasks. By conducting the ablation study, it is confirmed that the proposed RL with contrastive learning overcomes policy collapse. Supplementary demonstrations are available at https://cyberyyc.github.io/.
Read the full abstract

All authors Yicheng Yang, Ruijiao Li, Lifeng Wang, Shuai Zheng, Shunzheng Ma, Keyu Zhang, Tuoyu Sun, Chenyun Dai, Jie Ding, Zhuo Zou

Robotics · Cross-listed from cs.AI

RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Driving

Ruturaj Reddy, Hrishav Bakul Barua, Junn Yong Loo et al. (+2 more)

Diffusion-based trajectory planners can model multi-modal driving behavior, but their iterative denoising process introduces a latency bottleneck for real-time closed-loop deployment. We present RAPiD, a reward-guided consistency distillation framework that distills a pretrained DiffusionPlanner into a few-step consistency student while retaining multi-modal trajectory generation. The student is trained using deterministic teacher denoising steps from the frozen diffusion planner, together with a low-noise data anchor that keeps generated trajectories grounded in expert demonstrations. To make distillation safety-aware, we train an Implicit Q-Learning critic on a balanced mixture of ground-truth log-replay and DiffusionPlanner rollout trajectories, each scored using a modified PDM-style reward, providing trajectory-level supervision beyond conventional imitation learning. During deployment, the 2-step student generates K trajectories, and the trained critic performs best-of-K trajectory selection conditioned on the latent state. On nuPlan, RAPiD maintains comparable performance to the diffusion teacher on non-reactive closed-loop splits and remains competitive on reactive splits, while reducing complete-pipeline inference latency from 100.91 ms to 18.41 ms, corresponding to a 5.5x speedup. On interPlan, RAPiD achieves the highest aggregate score among learning-based methods, demonstrating competitive generalization in interactive long-tail scenarios. These results show that reward-guided consistency distillation can convert a pretrained diffusion planner into a few-step closed-loop planner that substantially reduces inference cost while preserving safety-oriented trajectory selection. The official website of this work is: https://github.com/ruturajreddy/RAPiD
Read the full abstract

All authors Ruturaj Reddy, Hrishav Bakul Barua, Junn Yong Loo, Thanh Thi Nguyen, Ganesh Krishnasamy

Robotics

Why Look at It at All?: Vision-Free Multifingered Blind Grasping Using Uniaxial Fingertip Force Sensing

Edgar Lee, Junho Choi, Taemin Kim et al. (+2 more)

Grasping under limited sensing remains a fundamental challenge for real-world robotic manipulation, as vision and high-resolution tactile sensors often introduce cost, fragility, and integration complexity. This work demonstrates that reliable multifingered grasping can be achieved under extremely minimal sensing by relying solely on uniaxial fingertip force feedback and joint proprioception, without vision or multi-axis/tactile sensing. To enable such blind grasping, we employ an efficient teacher-student training pipeline in which a reinforcement-learned teacher exploits privileged simulation-only observations to generate demonstrations for distilling a transformer-based student policy operating under partial observation. The student policy is trained to act using only sensing modalities available at real-world deployment. We validate the proposed approach on real hardware across 18 objects, including both in-distribution and out-of-distribution cases, achieving a 98.3~$\%$ overall grasp success rate. These results demonstrate strong robustness and generalization beyond the simulation training distribution, while significantly reducing sensing requirements for real-world grasping systems.
Read the full abstract

All authors Edgar Lee, Junho Choi, Taemin Kim, Changjoo Nam, Seokhwan Jeong

Robotics

Action-to-Action Flow Matching

Jindou Jia, Gen Li, Xiangyu Chen et al. (+5 more)

Diffusion-based policies have recently achieved remarkable success in robotics by formulating action prediction as a conditional denoising process. However, the standard practice of sampling from random Gaussian noise often requires multiple iterative steps to produce clean actions, leading to high inference latency that incurs a major bottleneck for real-time control. In this paper, we challenge the necessity of uninformed noise sampling and propose Action-to-Action flow matching (A2A), a novel policy paradigm that shifts from random sampling to initialization informed by the previous proprioceptive action. Unlike existing methods that treat proprioceptive action feedback as static conditions, A2A leverages historical proprioceptive sequences, embedding them into a high-dimensional latent space as the starting point for action generation. This design bypasses costly iterative denoising while effectively capturing the robot's physical dynamics and temporal continuity. Extensive experiments demonstrate that A2A exhibits high training efficiency, fast inference speed, and improved generalization. Notably, A2A enables high-quality action generation in as few as a single inference step, and exhibits superior robustness to visual perturbations and enhanced generalization to unseen configurations. Lastly, we also extend A2A to video generation, demonstrating its broader versatility in temporal modeling. Project site: https://lorenzo-0-0.github.io/A2A_Flow_Matching.
Read the full abstract

All authors Jindou Jia, Gen Li, Xiangyu Chen, Tuo An, Yuxuan Hu, Jingliang Li, Xinying Guo, Jianfei Yang

Friday 6 February

Robotics

Optimization of an Augmented R-CUBE mechanism for Cervical Surgery

Terence Essomba, Yu-Wen Wu, Abdelbadia Chaker et al. (+1 more)

In some surgical operations targeting the spine, it is required to drill cavities in the vertebrae for the insertion of pedicle screws. A new mechanical architecture is proposed for this application. It is based on an augmented version of the full translational R-CUBE mechanism, with improved linkages to implement additional rotational motion. Using this concept, a mechanism presented with a 3T2R motion that is required for the manipulation of the surgical drill. It is mainly composed three stages: one translational, one transmitting and one rotational. Their respective kinematic and velocity models are separately derived, then combined. Based on the drilling trajectories obtained from a real patient case, the mechanism is optimized for generating the highest kinematic performances.
Read the full abstract

All authors Terence Essomba, Yu-Wen Wu, Abdelbadia Chaker, Med Amine Laribi

Robotics

A novel Integrated Motion Tracking Device (IMTD) for Objective Laparoscopic Training Assessment: Development and Validation

Siwar Bouzid, Abdelbadia Chaker, Marc Arsicault et al. (+2 more)

This paper presents a novel, compact four-degree-of-freedom motion-tracking device (IMTD) designed for training and evaluation in laparoscopic surgery. The device's kinematics, mechanical design, instrumentation, and prototypes are developed and presented to meet the specific requirements of laparoscopic training context, including movement around a fixed center of motion and seamless integration into standard box trainers. The system IMTD's tracking accuracy and reliability are compared to a motion capture system (MoCap), assessing its ability to capture both angular and translational motions of surgical instruments. The study then focuses on key performance parameters including precision, fluidity, speed, and overall motion efficiency. The results highlight the system's effectiveness in tracking surgical gestures, providing valuable insights into its potential as a tool for training and performance evaluation in minimally invasive surgery. Additionally, IMTD's low cost and integrated design allow for easy integration and implementation in training rooms, offering a practical and accessible solution for general use. By offering objective, real-time feedback, the system can significantly contribute to improving surgical skills and shortening the learning curve for novice students, while also providing a foundation for future development of gesture scoring algorithms and standardized training protocols.
Read the full abstract

All authors Siwar Bouzid, Abdelbadia Chaker, Marc Arsicault, Sami Bennour, Med Amine Laribi

Robotics

Feasible Static Workspace Optimization of Tendon Driven Continuum Robot based on Euclidean norm

Mohammad Jabari, Carmen Visconte, Giuseppe Quaglia et al. (+1 more)

This paper focuses on the optimal design of a tendon-driven continuum robot (TDCR) based on its feasible static workspace (FSW). The TDCR under consideration is a two-segment robot driven by eight tendons, with four tendon actuators per segment. Tendon forces are treated as design variables, while the feasible static workspace (FSW) serves as the optimization objective. To determine the robot's feasible static workspace, a genetic algorithm optimization approach is employed to maximize a Euclidian norm of the TDCR's tip position over the workspace. During the simulations, the robot is subjected to external loads, including torques and forces. The results demonstrate the effectiveness of the proposed method in identifying optimal tendon forces to maximize the feasible static workspace, even under the influence of external forces and torques.
Read the full abstract

All authors Mohammad Jabari, Carmen Visconte, Giuseppe Quaglia, Med Amine Laribi

Robotics

aerial-autonomy-stack – a Faster-than-real-time, Autopilot-agnostic, ROS2 Framework to Simulate and Deploy Perception-based Drones

Jacopo Panerati, Sina Sajjadi, Sina Soleymanpour et al. (+2 more)

Unmanned aerial vehicles are rapidly transforming multiple applications, from agricultural and infrastructure monitoring to logistics and defense. Introducing greater autonomy to these systems can simultaneously make them more effective as well as reliable. Thus, the ability to rapidly engineer and deploy autonomous aerial systems has become of strategic importance. In the 2010s, a combination of high-performance compute, data, and open-source software led to the current deep learning and AI boom, unlocking decades of prior theoretical work. Robotics is on the cusp of a similar transformation. However, physical AI faces unique hurdles, often combined under the umbrella term "simulation-to-reality gap". These span from modeling shortcomings to the complexity of vertically integrating the highly heterogeneous hardware and software systems typically found in field robots. To address the latter, we introduce aerial-autonomy-stack, an open-source, end-to-end framework designed to streamline the pipeline from (GPU-accelerated) perception to (flight controller-based) action. Our stack allows the development of aerial autonomy using ROS2 and provides a common interface for two of the most popular autopilots: PX4 and ArduPilot. We show that it supports over 20x faster-than-real-time, end-to-end simulation of a complete development and deployment stack – including edge compute and networking – significantly compressing the build-test-release cycle of perception-based autonomy.
Read the full abstract

All authors Jacopo Panerati, Sina Sajjadi, Sina Soleymanpour, Varunkumar Mehta, Iraj Mantegh

Robotics

Realistic Synthetic Household Data Generation at Scale

Siddharth Singh, Ifrah Idrees and Abraham Dauhajre

Advancements in foundation models have catalyzed research in Embodied AI to develop interactive agents capable of environmental reasoning and interaction. Developing such agents requires diverse, large-scale datasets. Prior frameworks generate synthetic data for long-term human-robot interactions but fail to model the bidirectional influence between human behavior and household environments. Our proposed generative framework creates household datasets at scale through loosely coupled generation of long-term human-robot interactions and environments. Human personas influence environment generation, while environment schematics and semantics shape human-robot interactions. The generated 3D data includes rich static context such as object and environment semantics, and temporal context capturing human and agent behaviors over extended periods. Our flexible tool allows users to define dataset characteristics via natural language prompts, enabling configuration of environment and human activity data through natural language specifications. The tool creates variations of user-defined configurations, enabling scalable data generation. We validate our framework through statistical evaluation using multi-modal embeddings and key metrics: cosine similarity, mutual information gain, intervention analysis, and iterative improvement validation. Statistical comparisons show good alignment with real-world datasets (HOMER) with cosine similarity (0.60), while synthetic datasets (Wang et al.) show moderate alignment (0.27). Intervention analysis across age, organization, and sleep pattern changes shows statistically significant effects (p < 0.001) with large effect sizes (Cohen's d = 0.51-1.12), confirming bidirectional coupling translates persona traits into measurable environmental and behavioral differences. These contributions enable development and testing of household smart devices at scale.
Read the full abstract

Robotics · Cross-listed from cs.LG

Cerebellar-Inspired Residual Control for Fault Recovery: From Inference-Time Adaptation to Structural Consolidation

Nethmi Jayasinghe, Diana Gontero, Spencer T. Brown et al. (+3 more)

Robotic policies deployed in real-world environments often encounter post-training faults, where retraining, exploration, or system identification are impractical. We introduce an inference-time, cerebellar-inspired residual control framework that augments a frozen reinforcement learning policy with online corrective actions, enabling fault recovery without modifying base policy parameters. The framework instantiates core cerebellar principles, including high-dimensional pattern separation via fixed feature expansion, parallel microzone-style residual pathways, and local error-driven plasticity with excitatory and inhibitory eligibility traces operating at distinct time scales. These mechanisms enable fast, localized correction under post-training disturbances while avoiding destabilizing global policy updates. A conservative, performance-driven meta-adaptation regulates residual authority and plasticity, preserving nominal behavior and suppressing unnecessary intervention. Experiments on MuJoCo benchmarks under actuator, dynamic, and environmental perturbations show improvements of up to $+66\%$ on HalfCheetah-v5 and $+53\%$ on Humanoid-v5 under moderate faults, with graceful degradation under severe shifts and complementary robustness from consolidating persistent residual corrections into policy parameters.
Read the full abstract

All authors Nethmi Jayasinghe, Diana Gontero, Spencer T. Brown, Vinod K. Sangwan, Mark C. Hersam, Amit Ranjan Trivedi

Robotics

Continuum Robot Localization using Distributed Time-of-Flight Sensors

Spencer Teetaert, Giammarco Caroleo, Marco Pontin et al. (+4 more)

Localization and mapping of an environment are crucial tasks for any robot operating in unstructured environments. Time-of-flight (ToF) sensors (e.g.,~lidar) have proven useful in mobile robotics, where high-resolution sensors can be used for simultaneous localization and mapping. In soft and continuum robotics, however, these high-resolution sensors are too large for practical use. This, combined with the deformable nature of such robots, has resulted in continuum robot (CR) localization and mapping in unstructured environments being a largely untouched area. In this work, we present a localization technique for CRs that relies on small, low-resolution ToF sensors distributed along the length of the robot. By fusing measurement information with a robot shape prior, we show that accurate localization is possible despite each sensor experiencing frequent degenerate scenarios. We achieve an average localization error of 2.5cm in position and 7.2{\deg} in rotation across all experimental conditions with a 53cm long robot. We demonstrate that the results are repeated across multiple environments, in both simulation and real-world experiments, and study robustness in the estimation to deviations in the prior map.
Read the full abstract

All authors Spencer Teetaert, Giammarco Caroleo, Marco Pontin, Sven Lilge, Jessica Burgner-Kahrs, Timothy D. Barfoot, Perla Maiolino

Robotics

A compliant ankle-actuated compass walker with triggering timing control

Deniz Kerimoglu and Ismail Uyanik

Passive dynamic walkers are widely adopted as a mathematical model to represent biped walking. The stable locomotion of these models is limited to tilted surfaces, requiring gravitational energy. Various techniques, such as actuation through the ankle and hip joints, have been proposed to extend the applicability of these models to level ground and rough terrain with improved locomotion efficiency. However, most of these techniques rely on impulsive energy injection schemes and torsional springs, which are quite challenging to implement in a physical platform. Here, a new model is proposed, named triggering controlled ankle actuated compass gait (TC-AACG), which allows non-instantaneous compliant ankle pushoff. The proposed technique can be implemented in physical platforms via series elastic actuators (SEAs). Our systematic examination shows that the proposed approach extends the locomotion capabilities of a biped model compared to impulsive ankle pushoff approach. We provide extensive simulation analysis investigating the locomotion speed, mechanical cost of transport, and basin of attraction of the proposed model.
Read the full abstract

Robotics · Cross-listed from cs.CV

Zero-Shot UAV Navigation in Forests via Relightable 3D Gaussian Splatting

Zinan Lv, Yeqian Qian, Chen Sang et al. (+3 more)

UAV navigation in unstructured outdoor environments using passive monocular vision is hindered by the substantial visual domain gap between simulation and reality. While 3D Gaussian Splatting enables photorealistic scene reconstruction from real-world data, existing methods inherently couple static lighting with geometry, severely limiting policy generalization to dynamic real-world illumination. In this paper, we propose a novel end-to-end reinforcement learning framework designed for effective zero-shot transfer to unstructured outdoors. Within a high-fidelity simulation grounded in real-world data, our policy is trained to map raw monocular RGB observations directly to continuous control commands. To overcome photometric limitations, we introduce Relightable 3D Gaussian Splatting, which decomposes scene components to enable explicit, physically grounded editing of environmental lighting within the neural representation. By augmenting training with diverse synthesized lighting conditions ranging from strong directional sunlight to diffuse overcast skies, we compel the policy to learn robust, illumination-invariant visual features. Extensive real-world experiments demonstrate that a lightweight quadrotor achieves robust, collision-free navigation in complex forest environments at speeds up to 10 m/s, exhibiting significant resilience to drastic lighting variations without fine-tuning.
Read the full abstract

All authors Zinan Lv, Yeqian Qian, Chen Sang, Hao Liu, Danping Zou, Ming Yang

Robotics

Airspace-aware Contingency Landing Planning

H. Emre Tekaslan and Ella M. Atkins

This paper develops a real-time, search-based aircraft contingency landing planner that minimizes traffic disruptions while accounting for ground risk. The airspace model captures dense air traffic departure and arrival flows, helicopter corridors, and prohibited zones and is demonstrated with a Washington, D.C., area case study. Historical Automatic Dependent Surveillance-Broadcast (ADS-B) data are processed to estimate air traffic density. A low-latency computational geometry algorithm generates proximity-based heatmaps around high-risk corridors and restricted regions. Airspace risk is quantified as the cumulative exposure time of a landing trajectory within congested regions, while ground risk is assessed from overflown population density to jointly guide trajectory selection. A landing site selection module further mitigates disruption to nominal air traffic operations. Benchmarking against minimum-risk Dubins solutions demonstrates that the proposed planner achieves lower joint risk and reduced airspace disruption while maintaining real-time performance. Under airspace-risk-only conditions, the planner generates trajectories within an average of 2.9 seconds on a laptop computer. Future work will incorporate dynamic air traffic updates to enable spatiotemporal contingency landing planning that minimizes the need for real-time traffic rerouting.
Read the full abstract

Robotics

DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos

Shenyuan Gao, William Liang, Kaiyuan Zheng et al. (+27 more)

Being able to simulate the outcomes of actions in varied environments will revolutionize the development of generalist agents at scale. However, modeling these world dynamics, especially for dexterous robotics tasks, poses significant challenges due to limited data coverage and scarce action labels. As an endeavor towards this end, we introduce DreamDojo, a foundation world model that learns diverse interactions and dexterous controls from 44k hours of egocentric human videos. Our data mixture represents the largest video dataset to date for world model pretraining, spanning a wide range of daily scenarios with diverse objects and skills. To address the scarcity of action labels, we introduce continuous latent actions as unified proxy actions, enhancing interaction knowledge transfer from unlabeled videos. After post-training on small-scale target robot data, DreamDojo demonstrates a strong understanding of physics and precise action controllability. We also devise a distillation pipeline that accelerates DreamDojo to a real-time speed of 10.81 FPS and further improves context consistency. Our work enables several important applications based on generative world models, including live teleoperation, policy evaluation, and model-based planning. Systematic evaluation on multiple challenging out-of-distribution (OOD) benchmarks verifies the significance of our method for simulating open-world, contact-rich tasks, paving the way for general-purpose robot world models.
Read the full abstract

All authors Shenyuan Gao, William Liang, Kaiyuan Zheng, Ayaan Malik, Seonghyeon Ye, Sihyun Yu, Wei-Cheng Tseng, Yuzhu Dong, Kaichun Mo, Chen-Hsuan Lin, Qianli Ma, Seungjun Nah, Loic Magne, Jiannan Xiang, Yuqi Xie, Ruijie Zheng, Dantong Niu, You Liang Tan, K.R. Zentner, George Kurian, Suneel Indupuru, Pooya Jannaty, Jinwei Gu, Jun Zhang, Jitendra Malik, Pieter Abbeel, Ming-Yu Liu, Yuke Zhu, Joel Jang, Linxi "Jim" Fan