Robotics
Learning Ordinal Response Policies in Rank-Based Stochastic Prize-Collecting Games
Malintha Fernando, Petter Ögren and Silun Zhang
The Team Orienteering Problem (TOP) generalizes many real-world multi-
agent scheduling and routing tasks that occur in autonomous mobility, aerial logistics, and surveillance applications. While many
flavors of the TOP exist for planning in multi-
agent systems, they assume that all the
agents cooperate toward a single objective; therefore, they do not extend to settings when they compete in reward-scarce environments. We propose
Stochastic Prize-Collecting Orienteering Games (SPCOG) as an extension of the TOP to plan in the presence of self-interested
agents operating on a graph, under energy constraints and
stochastic transitions. A theoretical discussion on complete and
star graphs establishes that there is a unique pure Nash
equilibrium in SPCOGs that coincides with the optimal routing solution of an equivalent TOP under rank-based conflict resolution. We propose the concept of Ordinal Rank (OR) as a concise representation of an
agents' global rank and its location within a topological, well-defined neighborhood. Empirical evaluations conducted on real-world, road-network graphs under both dynamic and stationary prize distributions show that in parameter-sharing settings, the policies that leverage local information can outperform those policies leverage global information when the former is conditioned on the OR rather than the global rank, indicating that the OR acts as a strong inductive bias in multi-
agent games on graphs. The OR-conditioned policies also generalize much better to games with large number of
agents compared to global-rank conditioned policies. Finally, we also propose we propose Fictitious Ordinal Response Learning (FORL) as an
entropy-regulated
algorithm to obtain convergent policies in independent-learning settings in prize-collecting games on graphs.
Read the full abstract ▾
Robotics
Supervisory Measurement-Guided Noise Covariance Estimation
Haoying Li, Yifan Peng, Xinghan Li et al. (+1 more)
Reliable state estimation hinges on accurate specification of sensor noise covariances, which weigh heterogeneous measurements. In practice, these covariances are difficult to identify due to environmental variability, front-end preprocessing, and other reasons. We address this by formulating noise covariance estimation as a bilevel optimization that, from a
Bayesian perspective, factorizes the joint likelihood of so-called odometry and supervisory measurements, thereby balancing information utilization with computational efficiency. The factorization converts the nested
Bayesian dependency into a chain structure, enabling efficient parallel computation: at the lower level, an invariant extended Kalman filter with state augmentation estimates trajectories, while a derivative filter computes analytical gradients in parallel for upper-level gradient updates. The upper level refines the covariance to guide the lower-level estimation. Experiments on synthetic and real-world
datasets show that our method achieves higher efficiency over existing baselines.
Read the full abstract ▾
All authors Haoying Li, Yifan Peng, Xinghan Li, Junfeng Wu
Robotics · Cross-listed from cs.LG
Sample-efficient and Scalable Exploration in Continuous-Time RL
Klemens Iten, Lenart Treven, Bhavya Sukhija et al. (+2 more)
Reinforcement learning
algorithms are typically designed for discrete-time dynamics, even though the underlying real-world control systems are often continuous in time. In this paper, we study the problem of continuous-time reinforcement learning, where the unknown system dynamics are represented using nonlinear ordinary differential equations (ODEs). We leverage probabilistic models, such as Gaussian processes and
Bayesian neural networks, to learn an uncertainty-aware model of the underlying ODE. Our
algorithm, COMBRL, greedily maximizes a weighted sum of the extrinsic reward and model epistemic uncertainty. This yields a scalable and sample-efficient approach to continuous-time model-based RL. We show that COMBRL achieves sublinear regret in the reward-driven setting, and in the unsupervised RL setting (i.e., without extrinsic rewards), we provide a sample complexity bound. In our experiments, we evaluate COMBRL in both standard and unsupervised RL settings and demonstrate that it scales better, is more sample-efficient than prior methods, and outperforms baselines across several deep RL tasks.
Read the full abstract ▾
All authors Klemens Iten, Lenart Treven, Bhavya Sukhija, Florian Dörfler, Andreas Krause
Robotics · Cross-listed from cs.AI
Adaptive Surrogate Gradients for Sequential Reinforcement Learning in Spiking Neural Networks
Korneel Van den Berghe, Stein Stroobants, Vijay Janapa Reddi et al. (+1 more)
Neuromorphic computing systems are set to revolutionize energy-constrained robotics by achieving orders-of-magnitude efficiency gains, while enabling native temporal processing. Spiking Neural Networks (SNNs) represent a promising algorithmic approach for these systems, yet their application to complex control tasks faces two critical challenges: (1) the non-differentiable nature of spiking
neurons necessitates surrogate gradients with unclear optimization properties, and (2) the stateful dynamics of SNNs require training on sequences, which in reinforcement learning (RL) is hindered by limited sequence lengths during early training, preventing the network from bridging its warm-up period. We address these challenges by systematically analyzing surrogate gradient slope settings, showing that shallower slopes increase gradient magnitude in deeper layers but reduce
alignment with true gradients. In supervised learning, we find no clear preference for fixed or scheduled slopes. The effect is much more pronounced in RL settings, where shallower slopes or scheduled slopes lead to a 2.1x improvement in both training and final deployed performance. Next, we propose a novel training approach that leverages a privileged guiding policy to bootstrap the learning process, while still exploiting online environment interactions with the spiking policy. Combining our method with an adaptive slope schedule for a real-world drone position control task, we achieve an average return of 400 points, substantially outperforming prior techniques, including Behavioral Cloning and TD3BC, which achieve at most –200 points under the same conditions. This work advances both the theoretical understanding of surrogate gradient learning in SNNs and practical training methodologies for neuromorphic controllers demonstrated in real-world robotic systems.
Read the full abstract ▾
All authors Korneel Van den Berghe, Stein Stroobants, Vijay Janapa Reddi, G.C.H.E. de Croon
Robotics
Flatness-based trajectory planning for 3D overhead cranes with friction compensation and collision avoidance
Jorge Vicente-Martinez and Edgar Ramirez-Laboreo
This paper presents an optimal trajectory generation method for 3D overhead cranes by leveraging differential flatness. This framework enables the direct inclusion of complex physical and dynamic constraints, such as nonlinear friction and collision avoidance for both payload and rope. Our approach allows for aggressive movements by constraining payload swing only at the final point. A comparative simulation study validates our approach, demonstrating that neglecting dry friction leads to actuator saturation and collisions. The results show that friction modeling is a fundamental requirement for fast and safe crane trajectories.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
GenTrack2: An Improved Hybrid Approach for Multi-Object Tracking
Toan Van Nguyen, Rasmus G. K. Christiansen, Dirk Kraft et al. (+1 more)
This paper proposes a visual multi-object tracking method that jointly employs
stochastic and deterministic mechanisms to ensure identifier consistency for unknown and time-varying target numbers under nonlinear dynamics. A
stochastic particle filter addresses nonlinear dynamics and non-Gaussian noise, with support from particle swarm optimization (PSO) to guide particles toward state distribution modes and mitigate divergence through proposed fitness measures incorporating motion consistency, appearance similarity, and social-interaction cues with neighboring targets. Deterministic association further enforces identifier consistency via a proposed cost
matrix incorporating spatial consistency between particles and current detections, detection confidences, and track penalties. Subsequently, a novel scheme is proposed for the smooth updating of target states while preserving their identities, particularly for weak tracks during interactions with other targets and prolonged occlusions. Moreover, velocity regression over past states provides trend-seed velocities, enhancing particle sampling and state updates. The proposed tracker is designed to operate flexibly for both pre-recorded
videos and camera live streams, where future frames are unavailable. Experimental results confirm superior performance compared to state-of-the-art trackers. The source-code reference implementations of both the proposed method and compared-trackers are provided on GitHub: https://github.com/SDU-VelKoTek/GenTrack2
Read the full abstract ▾
All authors Toan Van Nguyen, Rasmus G. K. Christiansen, Dirk Kraft, Leon Bodenhagen
Robotics · Cross-listed from cs.CV
GenTrack: A New Generation of Multi-Object Tracking
Toan Van Nguyen, Rasmus G. K. Christiansen, Dirk Kraft et al. (+1 more)
This paper introduces a novel multi-object tracking (MOT) method, dubbed GenTrack, whose main contributions include: first-a hybrid tracking approach employing both
stochastic and deterministic manners to robustly handle unknown and time-varying numbers of targets, particularly in maintaining target identity (ID) consistency and managing nonlinear dynamics, second-leveraging particle swarm optimization (PSO) with some proposed fitness measures to guide
stochastic particles toward their target distribution modes, enabling effective tracking even with weak and noisy object detectors, third-integration of social interactions among targets to enhance PSO-guided particles as well as improve continuous updates of both strong (matched) and weak (unmatched) tracks, thereby reducing ID switches and track loss, especially during occlusions, fourth-a GenTrack-based redefined visual MOT baseline incorporating a comprehensive state and observation model based on space consistency, appearance, detection confidence, track penalties, and social scores for systematic and efficient target updates, and five-the first ever publicly available source-code reference implementation with minimal dependencies, featuring three variants, including GenTrack Simple, Strengthen, and Super, facilitating flexible reimplementation. Experimental results have shown that GenTrack provides superior performance on standard
benchmarks and real-world scenarios compared to state-of-the-art trackers, with integrated implementations of baselines for fair comparison. Potential directions for future work are also discussed. The source-code reference implementations of both the proposed method and compared-trackers are provided on GitHub: https://github.com/SDU-VelKoTek/GenTrack
Read the full abstract ▾
All authors Toan Van Nguyen, Rasmus G. K. Christiansen, Dirk Kraft, Leon Bodenhagen
Robotics
NVSim: Novel View Synthesis Simulator for Large Scale Indoor Navigation
Mingyu Jeong, Eunsung Kim, Sehun Park et al. (+1 more)
We present NVSim, a framework that automatically constructs large-scale, navigable indoor simulators from only common image sequences, overcoming the cost and scalability limitations of traditional 3D scanning. Our approach adapts 3D Gaussian Splatting to address visual artifacts on sparsely observed floors a common issue in robotic traversal data. We introduce Floor-Aware Gaussian Splatting to ensure a clean, navigable ground plane, and a novel mesh-free traversability checking
algorithm that constructs a topological graph by directly analyzing rendered views. We demonstrate our system's ability to generate valid, large-scale
navigation graphs from real-world data. A
video demonstration is avilable at https://youtu.be/tTiIQt6nXC8
Read the full abstract ▾
All authors Mingyu Jeong, Eunsung Kim, Sehun Park, Andrew Jaeyong Choi
Robotics
Global-State-Free Obstacle Avoidance for Quadrotor Control in Air-Ground Cooperation
Baozhe Zhang, Xinwei Chen, Qingcheng Chen et al. (+3 more)
CoNi-MPC provides an efficient framework for
UAV control in air-ground cooperative tasks by relying exclusively on relative states, eliminating the need for global state estimation. However, its lack of environmental information poses significant challenges for obstacle avoidance. To address this issue, we propose a novel obstacle avoidance
algorithm, Cooperative Non-inertial frame-based Obstacle Avoidance (CoNi-OA), designed explicitly for
UAV-UGV cooperative scenarios without reliance on global state estimation or obstacle prediction. CoNi-OA uniquely utilizes a single frame of raw
LiDAR data from the
UAV to generate a modulation
matrix, which directly adjusts the quadrotor's velocity to achieve obstacle avoidance. This modulation-based method enables real-time generation of collision-free trajectories within the UGV's non-inertial frame, significantly reducing computational demands (less than 5 ms per iteration) while maintaining safety in dynamic and unpredictable environments. The key contributions of this work include: (1) a modulation-based obstacle avoidance
algorithm specifically tailored for
UAV-UGV cooperation in non-inertial frames without global states; (2) rapid, real-time trajectory generation based solely on single-frame
LiDAR data, removing the need for obstacle modeling or prediction; and (3) adaptability to both static and dynamic environments, thus extending applicability to featureless or unknown scenarios.
Read the full abstract ▾
All authors Baozhe Zhang, Xinwei Chen, Qingcheng Chen, Chao Xu, Fei Gao, Yanjun Cao
Robotics
DynaRend: Learning 3D Dynamics via Masked Future Rendering for Robotic Manipulation
Jingyi Tian, Le Wang, Sanping Zhou et al. (+3 more)
Learning generalizable robotic manipulation policies remains a key challenge due to the scarcity of diverse real-world training data. While recent approaches have attempted to mitigate this through self-supervised representation learning, most either rely on 2D
vision pretraining paradigms such as masked image modeling, which primarily focus on static semantics or scene geometry, or utilize large-scale
video prediction models that emphasize 2D dynamics, thus failing to jointly learn the geometry, semantics, and dynamics required for effective manipulation. In this paper, we present DynaRend, a representation learning framework that learns 3D-aware and dynamics-informed triplane features via masked reconstruction and future prediction using differentiable volumetric rendering. By pretraining on multi-view RGB-D
video data, DynaRend jointly captures spatial geometry, future dynamics, and task semantics in a unified triplane representation. The learned representations can be effectively transferred to downstream robotic manipulation tasks via action value map prediction. We evaluate DynaRend on two challenging
benchmarks, RLBench and Colosseum, as well as in real-world robotic experiments, demonstrating substantial improvements in policy success rate, generalization to environmental perturbations, and real-world applicability across diverse manipulation tasks.
Read the full abstract ▾
All authors Jingyi Tian, Le Wang, Sanping Zhou, Sen Wang, Jiayi Li, Gang Hua
Robotics · Cross-listed from cs.CL
Can LLMs Translate Human Instructions into a Reinforcement Learning Agent's Internal Emergent Symbolic Representation?
Ziqi Ma, Sao Mai Nguyen and Philippe Xu
Emergent symbolic representations are critical for enabling developmental learning
agents to plan and generalize across tasks. In this work, we investigate whether large
language models (LLMs) can translate human natural
language instructions into the internal symbolic representations that emerge during hierarchical reinforcement learning. We apply a structured evaluation framework to measure the translation performance of commonly seen LLMs – GPT, Claude, Deepseek and Grok – across different internal symbolic partitions generated by a hierarchical reinforcement learning
algorithm in the Ant Maze and Ant Fall environments. Our findings reveal that although LLMs demonstrate some ability to translate natural
language into a symbolic representation of the environment dynamics, their performance is highly sensitive to partition granularity and task complexity. The results expose limitations in current LLMs capacity for representation
alignment, highlighting the need for further research on robust
alignment between
language and internal
agent representations.
Read the full abstract ▾
Robotics
Manipulate as Human: Learning Task-oriented Manipulation Skills by Adversarial Motion Priors
Ziqi Ma, Changda Tian and Yue Gao
In recent years, there has been growing interest in developing
robots and autonomous systems that can interact with human in a more natural and intuitive way. One of the key challenges in achieving this goal is to enable these systems to manipulate objects and tools in a manner that is similar to that of humans. In this paper, we propose a novel approach for learning human-style manipulation skills by using
adversarial motion priors, which we name HMAMP. The approach leverages
adversarial networks to model the complex dynamics of tool and object manipulation, as well as the aim of the manipulation task. The discriminator is trained using a combination of real-world data and simulation data executed by the
agent, which is designed to train a policy that generates realistic motion trajectories that match the statistical properties of human motion. We evaluated HMAMP on one challenging manipulation task: hammering, and the results indicate that HMAMP is capable of learning human-style manipulation skills that outperform current baseline methods. Additionally, we demonstrate that HMAMP has potential for real-world applications by performing real
robot arm hammering tasks. In general, HMAMP represents a significant step towards developing
robots and autonomous systems that can interact with humans in a more natural and intuitive way, by learning to manipulate tools and objects in a manner similar to how humans do.
Read the full abstract ▾
Robotics
Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogames
Ev Zisselman, Mirco Mutti, Shelly Francis-Meretzki et al. (+2 more)
Behavioral cloning is a simple yet effective technique for learning sequential decision-making from demonstrations. Recently, it has gained prominence as the core of foundation models for the physical world, where achieving generalization requires countless demonstrations of a multitude of tasks. Typically, a human expert with full information on the task demonstrates a (nearly) optimal behavior. In this paper, we propose to hide some of the task's information from the demonstrator. This “blindfolded” expert is compelled to employ non-trivial exploration to solve the task. We show that cloning the blindfolded expert generalizes better to unseen tasks than its fully-informed counterpart. We conduct experiments of real-world
robot peg insertion tasks with (limited) human demonstrations, alongside videogames from the Procgen
benchmark. Additionally, we support our findings with theoretical analysis, which confirms that the generalization error scales with $\sqrt{I/m}$, where $I$ measures the amount of task information available to the demonstrator, and $m$ is the number of demonstrated tasks. Both theory and practice indicate that cloning blindfolded experts generalizes better with fewer demonstrated tasks. Project page with
videos and code: https://sites.google.com/view/blindfoldedexperts/home
Read the full abstract ▾
All authors Ev Zisselman, Mirco Mutti, Shelly Francis-Meretzki, Elisei Shafer, Aviv Tamar
Robotics · Cross-listed from cs.AI
BLM$_1$: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning
Wentao Tan, Bowen Wang, Heng Zhi et al. (+15 more)
Multimodal large
language models (MLLMs) have advanced
vision-
language reasoning and are increasingly deployed in
embodied agents. However, significant limitations remain: MLLMs generalize poorly across digital-physical spaces and embodiments;
vision-
language-action models (VLAs) produce low-level actions yet lack robust high-level
embodied reasoning; and most
embodied large
language models (ELLMs) are constrained to digital-space with poor generalization to the physical world. Thus, unified models that operate seamlessly across digital and physical spaces while generalizing across embodiments and tasks remain absent. We introduce the \textbf{Boundless Large Model (BLM$_1$)}, a
multimodal spatial foundation model that preserves instruction following and
reasoning, incorporates
embodied knowledge, and supports robust cross-embodiment control. BLM$_1$ integrates three key capabilities – cross-space transfer, cross-task learning, and cross-embodiment generalization – via a two-stage training paradigm. Stage I injects
embodied knowledge into the MLLM through curated digital corpora while maintaining
language competence. Stage II trains a policy module through an intent-bridging interface that extracts high-level semantics from the MLLM to guide control, without fine-tuning the MLLM backbone. This process is supported by a self-collected cross-embodiment demonstration suite spanning four
robot embodiments and six progressively challenging tasks. Evaluations across digital and physical
benchmarks show that a single BLM$_1$ instance outperforms four model families – MLLMs, ELLMs, VLAs, and GMLMs – achieving $\sim\!\textbf{6%}$ gains in digital tasks and $\sim\!\textbf{3%}$ in physical tasks.
Read the full abstract ▾
All authors Wentao Tan, Bowen Wang, Heng Zhi, Chenyu Liu, Zhe Li, Jian Liu, Zengrong Lin, Yukun Dai, Yipeng Chen, Wenjie Yang, Enci Xie, Hao Xue, Baixu Ji, Chen Xu, Zhibin Wang, Tianshi Wang, Lei Zhu, Heng Tao Shen
Robotics
LagMemo: Language 3D Gaussian Splatting Memory for Multi-modal Open-vocabulary Multi-goal Visual Navigation
Haotian Zhou, Xiaole Wang, He Li et al. (+5 more)
Navigating to a designated goal using visual information is a fundamental capability for intelligent
robots. To address the practical demands of multi-modal, open-vocabulary goal queries and multi-goal visual
navigation, we propose LagMemo, a
navigation system that leverages a
language 3D Gaussian Splatting
memory. During a one-time exploration, LagMemo constructs a unified 3D
language memory with robust spatial-semantic correlations. With incoming task goals, the system efficiently queries the
memory, predicts candidate goal locations, and integrates a local perception-based verification mechanism to dynamically match and validate goals. For fair and rigorous evaluation, we curate GOAT-Core, a high-quality core split distilled from GOAT-Bench. Experimental results show that LagMemo's
memory module enables effective multi-modal open-vocabulary localization, and significantly outperforms state-of-the-art methods in multi-goal visual
navigation. Project page: https://weekgoodday.github.io/lagmemo
Read the full abstract ▾
All authors Haotian Zhou, Xiaole Wang, He Li, Zhuo Qi, Jinrun Yin, Haiyu Kong, Jianghuan Xu, Huijing Zhao
Robotics
PFEA: An LLM-based High-Level Natural Language Planning and Feedback Embodied Agent for Human-Centered AI
Wenbin Ding, Jun Chen, Mingjia Chen et al. (+3 more)
The rapid advancement of Large
Language Models (LLMs) has marked a significant breakthrough in Artificial Intelligence (AI), ushering in a new era of Human-centered Artificial Intelligence (HAI). HAI aims to better serve human welfare and needs, thereby placing higher demands on the intelligence level of
robots, particularly in aspects such as natural
language interaction, complex task planning, and execution. Intelligent
agents powered by LLMs have opened up new pathways for realizing HAI. However, existing LLM-based
embodied agents often lack the ability to plan and execute complex natural
language control tasks online. This paper explores the implementation of intelligent robotic manipulating
agents based on
Vision-
Language Models (VLMs) in the physical world. We propose a novel
embodied agent framework for
robots, which comprises a human-
robot voice interaction module, a
vision-
language agent module and an action execution module. The
vision-
language agent itself includes a
vision-based task planner, a natural
language instruction converter, and a task performance feedback evaluator. Experimental results demonstrate that our
agent achieves a 28% higher average task success rate in both simulated and real environments compared to approaches relying solely on LLM+CLIP, significantly improving the execution success rate of high-level natural
language instruction tasks.
Read the full abstract ▾
All authors Wenbin Ding, Jun Chen, Mingjia Chen, Fei Xie, Qi Mao, Philip Dames
Robotics
Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorer
Zhenxin Li, Nadine Chang, Wenhao Yao et al. (+9 more)
Human demonstrations are widely considered the cornerstone of end-to-end (E2E) autonomous driving despite human demonstration's scarcity for long-tail and safety-critical scenarios. Nonetheless, current E2E autonomous driving (AD) training paradigms continue to rely on human demonstrations. Imitation learning (IL) requires human demonstrations for training, whereas reinforcement learning (RL) has emerged as a promising alternative to reduce this dependency. However, most existing RL methods for E2E AD still rely implicitly on human demonstrations. A pure rewards-based RL method can overcome the need for human demonstrations, but general RL policy gradient methods suffer from the cold-start problem. In this paper, we propose ZTRS (Zero-human demonstration end-to-end autonomous driving with TRajectory Scorer) - a complete RL-based E2E planning paradigm trained solely on real-world images and rule-based rewards, entirely without human demonstration. Through our proposed Exhaustive Policy Optimization (EPO), a policy gradient variant tailored for enumerable trajectory actions and dense supervision, ZTRS enables the model to generalize better to long-tail driving scenarios. We demonstrate this generalization through our SOTA performance against IL approaches on both long-tail Navhard and closed-loop HUGSIM
datasets. Project page: https://zhenxinli.net/ZTRS/.
Read the full abstract ▾
All authors Zhenxin Li, Nadine Chang, Wenhao Yao, Xinglong Sun, Zi Wang, Maying Shen, Jingde Chen, Jingyu Song, Kailin Li, Zuxuan Wu, Shiyi Lan, Jose M. Alvarez
Robotics · Cross-listed from cs.LG
Learning Parameterized Skills from Demonstrations
Vedant Gupta, Haotian Fu, Calvin Luo et al. (+2 more)
We present DEPS, an end-to-end
algorithm for discovering parameterized skills from expert demonstrations. Our method learns parameterized skill policies jointly with a meta-policy that selects the appropriate discrete skill and continuous parameters at each timestep. Using a combination of temporal variational inference and information-theoretic regularization methods, we address the challenge of degeneracy common in latent variable models, ensuring that the learned skills are temporally extended, semantically meaningful, and adaptable. We empirically show that learning parameterized skills from multitask expert demonstrations significantly improves generalization to unseen tasks. Our method outperforms multitask as well as skill learning baselines on both LIBERO and MetaWorld
benchmarks. We also demonstrate that DEPS discovers interpretable parameterized skills, such as an object grasping skill whose continuous arguments define the grasp location.
Read the full abstract ▾
All authors Vedant Gupta, Haotian Fu, Calvin Luo, Yiding Jiang, George Konidaris
Robotics
Dynamically-Consistent Trajectory Optimization for Legged Robots via Contact Point Decomposition
Sangmin Kim, Hajun Kim, Gijeong Kim et al. (+2 more)
To generate reliable motion for legged
robots through trajectory optimization, it is crucial to simultaneously compute the
robot's path and contact sequence, as well as accurately consider the dynamics in the problem formulation. In this paper, we present a phase-based trajectory optimization that ensures the feasibility of translational dynamics and friction cone constraints throughout the entire trajectory. Specifically, our approach leverages the superposition properties of linear differential equations to decouple the translational dynamics for each contact point, which operates under different phase sequences. Furthermore, we utilize the differentiation
matrix of Bézier
polynomials to derive an analytical relationship between the
robot's position and force, thereby ensuring the consistent satisfaction of translational dynamics. Additionally, by exploiting the convex closure property of Bézier
polynomials, our method ensures compliance with friction cone constraints. Using the aforementioned approach, the proposed trajectory optimization framework can generate dynamically reliable motions with various gait sequences for legged
robots. We validate our framework using a quadruped
robot model, focusing on the feasibility of dynamics and motion generation. The source code and additional materials are publicly available at our project page: https://sangmin11.github.io/Analytical_Dynamics_TO_Project/.
Read the full abstract ▾
All authors Sangmin Kim, Hajun Kim, Gijeong Kim, Min-Gyu Kim, Hae-Won Park
Robotics
Balanced Collaborative Exploration via Distributed Topological Graph Voronoi Partition
Tianyi Ding, Ronghao Zheng, Senlin Zhang et al. (+1 more)
This work addresses the collaborative multi-
robot autonomous online exploration problem, particularly focusing on distributed exploration planning for dynamically balanced exploration area partition and task allocation among a team of mobile
robots operating in obstacle-dense non-convex environments. We present a novel topological map structure that simultaneously characterizes both spatial connectivity and global exploration completeness of the environment. The topological map is updated incrementally to utilize known spatial information for updating reachable spaces, while exploration targets are planned in a receding horizon fashion under global coverage guidance. A distributed weighted topological graph Voronoi
algorithm is introduced implementing balanced graph space partitions of the fused topological maps. Theoretical guarantees are provided for distributed consensus convergence and equitable graph space partitions with constant bounds. A local planner optimizes the visitation sequence of exploration targets within the balanced partitioned graph space to minimize travel distance, while generating safe, smooth, and dynamically feasible motion trajectories. Comprehensive benchmarking against state-of-the-art methods demonstrates significant improvements in exploration efficiency, completeness, and workload balance across the
robot team.
Read the full abstract ▾
All authors Tianyi Ding, Ronghao Zheng, Senlin Zhang, Meiqin Liu
Robotics
Language-Conditioned Representations and Mixture-of-Experts Policy for Robust Multi-Task Robotic Manipulation
Xiucheng Zhang, Yang Jiang, Hongwei Qing et al. (+1 more)
Perceptual ambiguity and task conflict limit multitask robotic manipulation via imitation learning. We propose a framework combining a
Language-Conditioned Visual Representation (LCVR) module and a
Language-conditioned Mixture-ofExperts Density Policy (LMoE-DP). LCVR resolves perceptual ambiguities by grounding visual features with
language instructions, enabling differentiation between visually similar tasks. To mitigate task conflict, LMoE-DP uses a sparse expert architecture to specialize in distinct,
multimodal action distributions, stabilized by gradient modulation. On real-
robot benchmarks, LCVR boosts Action Chunking with
Transformers (ACT) and
Diffusion Policy (DP) success rates by 33.75% and 25%, respectively. The full framework achieves a 79% average success, outperforming the advanced baseline by 21%. Our work shows that combining semantic grounding and expert specialization enables robust, efficient multi-task manipulation
Read the full abstract ▾
All authors Xiucheng Zhang, Yang Jiang, Hongwei Qing, Jiashuo Bai
Robotics
SynAD: Enhancing Real-World End-to-End Autonomous Driving Models through Synthetic Data Integration
Jongsuk Kim, Jaeyoung Lee, Gyojin Han et al. (+3 more)
Recent advancements in deep learning and the availability of high-quality real-world driving
datasets have propelled end-to-end autonomous driving. Despite this progress, relying solely on real-world data limits the variety of driving scenarios for training. Synthetic scenario generation has emerged as a promising solution to enrich the diversity of training data; however, its application within E2E AD models remains largely unexplored. This is primarily due to the absence of a designated ego vehicle and the associated sensor inputs, such as camera or
LiDAR, typically provided in real-world scenarios. To address this gap, we introduce SynAD, the first framework designed to enhance real-world E2E AD models using synthetic data. Our method designates the
agent with the most comprehensive driving information as the ego vehicle in a multi-
agent synthetic scenario. We further project path-level scenarios onto maps and employ a newly developed Map-to-BEV Network to derive bird's-eye-view features without relying on sensor inputs. Finally, we devise a training strategy that effectively integrates these map-based synthetic data with real driving data. Experimental results demonstrate that SynAD effectively integrates all components and notably enhances safety performance. By bridging synthetic scenario generation and E2E AD, SynAD paves the way for more comprehensive and robust autonomous driving models.
Read the full abstract ▾
All authors Jongsuk Kim, Jaeyoung Lee, Gyojin Han, Dongjae Lee, Minki Jeong, Junmo Kim
Robotics
Improved Accuracy of Robot Localization Using 3-D LiDAR in a Hippocampus-Inspired Model
Andrew Gerstenslager, Bekarys Dukenbaev and Ali A. Minai
Boundary Vector
Cells (BVCs) are a class of
neurons in the
brains of vertebrates that encode environmental boundaries at specific distances and allocentric directions, playing a central role in forming place fields in the hippocampus. Most computational BVC models are restricted to two-dimensional (2D) environments, making them prone to spatial ambiguities in the presence of horizontal
symmetries in the environment. To address this limitation, we incorporate vertical angular sensitivity into the BVC framework, thereby enabling robust boundary detection in three dimensions, and leading to significantly more accurate spatial localization in a biologically-inspired
robot model. The proposed model processes
LiDAR data to capture vertical contours, thereby disambiguating locations that would be indistinguishable under a purely 2D representation. Experimental results show that in environments with minimal vertical variation, the proposed 3D model matches the performance of a 2D baseline; yet, as 3D complexity increases, it yields substantially more distinct place fields and markedly reduces spatial aliasing. These findings show that adding a vertical dimension to BVC-based localization can significantly enhance
navigation and mapping in real-world 3D spaces while retaining performance parity in simpler, near-planar scenarios.
Read the full abstract ▾
Robotics
VOCALoco: Viability-Optimized Cost-aware Adaptive Locomotion
Stanley Wu, Mohamad H. Danesh, Simon Li et al. (+5 more)
Recent advancements in legged
robot locomotion have facilitated traversal over increasingly complex terrains. Despite this progress, many existing approaches rely on end-to-end deep reinforcement learning (DRL), which poses limitations in terms of safety and interpretability, especially when generalizing to novel terrains. To overcome these challenges, we introduce VOCALoco, a modular skill-selection framework that dynamically adapts locomotion strategies based on perceptual input. Given a set of pre-trained locomotion policies, VOCALoco evaluates their viability and energy-consumption by predicting both the safety of execution and the anticipated cost of transport over a fixed planning horizon. This joint assessment enables the selection of policies that are both safe and energy-efficient, given the observed local terrain. We evaluate our approach on staircase locomotion tasks, demonstrating its performance in both simulated and real-world scenarios using a quadrupedal
robot. Empirical results show that VOCALoco achieves improved robustness and safety during stair ascent and descent compared to a conventional end-to-end DRL policy
Read the full abstract ▾
All authors Stanley Wu, Mohamad H. Danesh, Simon Li, Hanna Yurchyk, Amin Abyaneh, Anas El Houssaini, David Meger, Hsiu-Chin Lin
Robotics
A Survey on Collaborative SLAM with 3D Gaussian Splatting
Phuc Nguyen Xuan, Thanh Nguyen Canh, Huu-Hung Nguyen et al. (+2 more)
This survey comprehensively reviews the evolving field of multi-
robot collaborative Simultaneous Localization and Mapping (SLAM) using 3D Gaussian Splatting (3DGS). As an explicit scene representation, 3DGS has enabled unprecedented real-time, high-fidelity rendering, ideal for robotics. However, its use in multi-
robot systems introduces significant challenges in maintaining global consistency, managing communication, and fusing data from heterogeneous sources. We systematically categorize approaches by their architecture – centralized, distributed – and analyze core components like multi-
agent consistency and
alignment, communication-efficient, Gaussian representation, semantic
distillation, fusion and pose optimization, and real-time scalability. In addition, a summary of critical
datasets and evaluation metrics is provided to contextualize performance. Finally, we identify key open challenges and chart future research directions, including lifelong mapping, semantic association and mapping, multi-model for robustness, and bridging the Sim2Real gap.
Read the full abstract ▾
All authors Phuc Nguyen Xuan, Thanh Nguyen Canh, Huu-Hung Nguyen, Nak Young Chong, Xiem HoangVan
Robotics
Adaptive-twist Soft Finger Mechanism for Grasping by Wrapping
Hiroki Ishikawa, Kyosuke Ishibashi and Ko Yamamoto
This paper presents a soft
robot finger capable of adaptive-twist deformation to grasp objects by wrapping them. For a soft hand to grasp and pick-up one object from densely contained multiple objects, a soft finger requires the adaptive-twist deformation function in both in-plane and out-of-plane directions. The function allows the finger to be inserted deeply into a limited gap among objects. Once inserted, the soft finger requires appropriate control of grasping force normal to contact surface, thereby maintaining the twisted deformation. In this paper, we refer to this type of grasping as grasping by wrapping. To achieve these two functions by a single actuation source, we propose a variable stiffness mechanism that can adaptively change the stiffness as the pressure is higher. We conduct a finite element analysis (FEA) on the proposed mechanism and determine its design parameter based on the FEA result. Using the developed soft finger, we report basic experimental results and demonstrations on grasping various objects.
Read the full abstract ▾
Robotics
A Comprehensive General Model of Tendon-Actuated Concentric Tube Robots with Multiple Tubes and Tendons
Pejman Kheradmand, Behnam Moradkhani, Raghavasimhan Sankaranarayanan et al. (+5 more)
Tendon-actuated concentric tube mechanisms combine the advantages of tendon-driven continuum
robots and concentric tube
robots while addressing their respective limitations. They overcome the restricted degrees of freedom often seen in tendon-driven designs, and mitigate issues such as snapping instability associated with concentric tube
robots. However, a complete and general mechanical model for these systems remains an open problem. In this work, we propose a Cosserat rod-based framework for modeling the general case of $n$ concentric tubes, each actuated by $m_i$ tendons, where $i = \{1, \ldots, n\}$. The model allows each tube to twist and elongate while enforcing a shared centerline for bending. We validate the proposed framework through experiments with two-tube and three tube assemblies under various tendon routing configurations, achieving tip prediction errors $<4\%$ of the
robot's total length. We further demonstrate the model's generality by applying it to existing
robots in the field, where maximum tip deviations remain around $5\%$ of the total length. This model provides a foundation for accurate shape estimation and control of advanced tendon-actuated concentric tube
robots.
Read the full abstract ▾
All authors Pejman Kheradmand, Behnam Moradkhani, Raghavasimhan Sankaranarayanan, Kent K. Yamamoto, Tanner J. Zachem, Patrick J. Codd, Yash Chitalia, Pierre E. Dupont