Robotics
OJOx: Specification-Conditioned Demonstrations for Embodied AI in Construction
Mohamed Dawod
Large-scale egocentric and whole-body human demonstrations are becoming a primary source of data for embodied intelligence. They record what people perceive and do, but rarely the external specification that gave an action its purpose. In construction that omission is consequential: skilled work is directed at project-specific configurations defined in a design model - configurations not yet present in the environment being observed. A mason's transferable competence is not the geometry of one wall but the ability to realise a new geometry from a specification. We introduce the specification-conditioned demonstration: a synchronised record of the physical state a demonstrator perceives, the intended state supplied to them by an external design, and the behaviour connecting the two. We present OJOx, a capture interface that realises this for construction - delivering design geometry to a headset, anchoring it in the physical workspace, rendering it into a demonstrator's stereo passthrough view, and recording that view synchronously with whole-body and hand motion. We report one fully instrumented session - a 33-component wall laid against a specification that changes while the work proceeds - and check the record against the physical scene through an external camera registered independently of the capture. Recorded sessions remain compatible with existing humanoid retargeting infrastructure and replay onto a Unitree G1 in simulation. The result is a data interface for testing whether embodied policies can learn not merely to imitate demonstrated actions, but to act toward specifications absent from their training experience.
Read the full abstract ▾
Robotics
ORDER: A Fictitious-World Benchmark for Domain-Adaptive Embodied AI
Sai Krishna Reddy Sathi and Anuj Tiwari
Adapting
language models to new domains via continual pre-training raises a basic evaluation problem: if the training corpus overlaps with what the model already knows, performance gains cannot be cleanly attributed to new learning rather than pre-existing knowledge. This matters most for knowledge-intensive, task-light (KHTL)
robot deployments - pharmaceutical dispensing, hazardous-material handling, facility-specific protocols, where the physical task is simple but the governing rules are proprietary and safety-critical, and where extensive live testing is costly or unsafe. We introduce ORDER (Ontology-driven Decision-making for Embodied
Reasoning), a
benchmark built on a fictitious world: a 342,069-
token synthetic corpus defining a self-consistent physics that cannot appear in any model's pre-training data. ORDER pairs a 500-question knowledge test (ORDER-BENCH) with a harder compositional task, ORDER-SPATIAL: ordering objects for safe manipulation across both familiar and entirely novel scenes. GPT-4.1 without adaptation scores below chance on ORDER-SPATIAL (Kendall's tau = 0.441), showing its priors actively conflict with the invented physics. After continual pre-training, small models improve substantially on both familiar and novel scenes alike evidence of genuine world-model induction rather than memorization. We then carry this through to a
robot pipeline: models that answer the knowledge test well often cannot produce valid, executable plans without a further skill-adaptation stage, after which small, fully offline models outperform GPT-4.1 even when GPT-4.1 is given
retrieval access to the same rules (Kendall's tau = 0.848 vs. 0.606), on a full perception-to-execution loop demonstrated on a simulated iiwa7 arm with human-in-the-loop correction. Throughout, ORDER-SPATIAL performance, not knowledge-test accuracy is what predicts real plan quality.
Read the full abstract ▾
Robotics
AssemblyGrid v1: A Benchmark for Multi-Robot Production with Temporary Coalitions, Local Information, and Geometric Constraints
Fouad Bahrpeyma, David Heik and Dirk Reichelt
Flexible robotic production requires joint decisions on process progression, material routing, resource assignment, temporary cooperation, and simultaneous execution, since each decision can affect the feasibility of the others. The challenge is greater under decentralized control, where each
robot acts from bounded local information while system progress depends on collective decisions, shared resources, material state, and workspace compatibility. These properties closely match cooperative multi-
agent decision making under partial observability and resource contention. This paper introduces AssemblyGrid v1, a reproducible
benchmark for repeated multi-
robot production that combines explicit process progression, decentralized observations, material transfer, temporary multi-
robot coalitions, productive concurrency, and geometry-dependent feasibility within one task-level formulation. The
benchmark includes Flow, Coalition, and Concurrency workload families, each with three scenario levels. Task success and evaluation measures are defined independently of learning reward and solution method, allowing learning-based and non-learning methods to address the same production problem. AssemblyGrid v1 is evaluated through executable conformance checks, mechanism studies, and algorithmic experiments using a privileged centralized reference, structured decentralized controllers, and MARL methods including IPPO, MAPPO, and QMIX. Results demonstrate productive execution under centralized and decentralized control. The MARL experiments further show that decentralized policies can learn effective production behavior from local observations and actions, supporting AssemblyGrid as a controlled
benchmark for studying cooperative decision making in flexible robotic production.
Read the full abstract ▾
Robotics
World-Action Models for Robot Learning and Control: A Survey
Zuxing Lu, Hongjia Zhai, Guanzhi Wang et al. (+13 more)
Robots operating in open environments act under partial observability, physical constraints, and dynamic task contexts. Beyond mapping observations and
language instructions to actions, they must anticipate how candidate actions may affect future states and task-relevant outcomes. Recent advances in world models,
video generation, and
Vision-
Language-Action (VLA) policies have motivated the development of World-Action Models (WAMs), which couple future world prediction with executable action generation. This survey provides a robotics-oriented review of WAMs. We clarify their scope relative to conventional world models, model-based reinforcement learning, action-conditioned
video generation, and reactive VLA policies, and organize existing methods through a unified taxonomy covering representations, transition modeling, action interfaces, architectures, training pipelines, data modalities, and scaling strategies. We further review applications of WAMs in manipulation, navigation, and autonomous driving, and we summarize the
datasets,
benchmarks, metrics, and protocols used to evaluate WAM systems. Finally, we discuss key challenges in action
alignment, world-action factorization, spatial and multi-view consistency, long-horizon
memory, neural simulation for closed-loop policy learning, and efficient inference. Taken together, this survey aims to provide a concise technical foundation for integrating predictive world modeling with action generation, toward more reliable embodied
robot intelligence. Project page: https://rcl-robotics.github.io/Awesome-World-Action-Models.
Read the full abstract ▾
All authors Zuxing Lu, Hongjia Zhai, Guanzhi Wang, Huajian Zeng, Jiaqi Yang, Jingyu Liu, Lei Cheng, Yuantai Zhang, Yuheng Qiu, Zezhou Cheng, Ivan Laptev, Danfei Xu, Benjamin Riviere, Giuseppe Loianno, Eric Xing, Xingxing Zuo
Robotics
Belief-Adaptive Online Autonomy for Quadrotor UAV Navigation under GNSS Degradation in Urban Environments
Deepak Kumar Panda and Weisi Guo
Reliable online autonomy is critical for quadrotor operation in urban airspaces, where global navigation satellite systems (GNSS) measurements suffer from multipath, blockage, and latency issues, introducing non-stationary, temporally correlated errors that degrade conventional GNSS-IMU fusion. This paper presents a belief-adaptive online autonomy framework that augments an extended Kalman filter (EKF) with explicit GNSS trust modelling, second-order online belief adaptation, and latency-aware out-of-sequence measurement handling. GNSS trust is represented as a latent belief state that modulates measurement weighting and multipath bias uncertainty, and is updated online using EKF consistency signals. Unlike reactive covariance tuning, the proposed approach enables proactive and stable sensor trust adaptation without prior environmental knowledge or offline training. Evaluation in simulated urban air mobility scenarios with correlated multipath,
stochastic latency, and obstacle constraints demonstrates improved belief convergence, smoother trajectories, and reduced estimation and tracking errors compared to naive, adaptive, and first-order baselines. The framework preserves classical GNSS-IMU fusion structure and can be integrated directly into existing flight control pipelines, supporting robust online autonomy in GNSS degraded environments.
Read the full abstract ▾
Robotics
Language-Guided Representation Learning for Robust Cross-Sensor Material Recognition
Mashood M. Mohsan, Muhayy Ud Din, Binzhao Xu et al. (+2 more)
Robots need touch to manipulate objects safely and reliably, as many properties, such as softness, texture, and contact stability, are hard to infer from
vision alone. However,
vision-based tactile sensors yield different observations of the same material due to variations in optics, elastomer properties, and illumination, leading to poor generalization when trained on a single or multiple sensors. We propose a
language-guided
distillation framework for learning sensor-robust tactile representations.
Language encodes high-level semantic properties of touch (e.g., rough, soft, slippery) that remain invariant across sensing hardware, providing a natural sensor-agnostic supervisory signal. We construct a 39K-sample touch-
language dataset with human-annotated material labels and train a tactile encoder to align sensor-specific tactile images with
language embeddings in a shared semantic space. We evaluate our approach for few-shot learning and cross-sensor transfer and
benchmark it on six existing tactile
datasets. Our method achieves 95% accuracy in the 100-shot setting, improves cross-sensor transfer by an average of 13.3% accuracy, and yields up to 19% accuracy gains across six existing tactile
datasets. These results demonstrate that
language-guided
distillation enables scalable and hardware-agnostic tactile representation learning. Code and
dataset are available at https://mashood3624.github.io/
Language_Tactile/
Read the full abstract ▾
All authors Mashood M. Mohsan, Muhayy Ud Din, Binzhao Xu, Ahmad Abubakar, Irfan Hussain
Robotics
A Personalized Dynamic Balance Evaluation Paradigm for Hip Exoskeleton-Assisted Walking under Unexpected Ground Perturbations
Yun Chen, Oluwasegun T. Akinniyi and Qiang Zhang
Hip exoskeletons may improve recovery from unexpected gait perturbations, yet personalizing assistance remains difficult because balance is multidimensional and human-in-the-loop experiments are small-sample and noisy. We present a participant-specific composite balance cost that integrates seven biomechanical sub-metrics spanning margin of stability, center-of-mass dynamics, and whole-body angular momentum. The sub-metrics are converted to direction-aligned, dimensionless cost features, and nonnegative fusion weights are learned on the simplex. Coupled with an empirical-Bayes hierarchical model, the learned-composite selector estimates each tested condition's posterior probability of being best, P(best), and a high-probability candidate set with size $K_{0.8}$. The framework was evaluated with three participants walking at 1.1 m/s during unilateral belt-slip perturbations across 46 hip-assistance conditions. In the full-budget analysis (B = 4 repeats per condition), the selector concentrated 80% of the posterior probability within 1 to 5 of 46 conditions, compared with 2 to 12 for equal-weight fusion and 4 to 37 for principal component analysis fusion. This smaller candidate set could shorten personalization experiments and limit participants' exposure to repeated perturbations in future studies. Selected-condition trials showed lower observed composite costs than no-torque trials, with nominal p < 0.05 for P2 and P3. Leave-one-repeat-out refits yielded positive mean held-out rank correlations for all participants and moderate stability of the learned weights and candidate sets. These proof-of-concept results support participant-specific composite balance evaluation for candidate selection in perturbation-based human-in-the-loop experiments.
Read the full abstract ▾
Robotics
An Adaptive Fixed-Time Line-of-Sight Guidance Scheme for 3D Path Following of Underwater Vehicles: Theory and Experiment
Hanzhi Yang, Jalil Chavez-Galaviz and Nina Mahmoudian
Reliable path tracking is crucial for autonomous underwater vehicles (AUVs) operating in dynamic and uncertain marine environments. However, traditional line-of-sight (LOS) guidance methods rely on asymptotic convergence, resulting in slow disturbance recovery and unpredictable tracking performance. Existing robust control methods typically require modifications to the underlying vehicle controller, limiting their practical application on commercial AUV platforms. This paper proposes a robust fixed-time adaptive LOS guidance framework for 3D path tracking for AUVs. By combining fixed-time stability theory with LOS guidance, this method guarantees path tracking convergence within a preset time range, with the convergence time independent of initial conditions. Furthermore, a fixed-time adaptive
estimator is developed to rapidly compensate for time-varying sideslip disturbances caused by ocean currents. A time-varying look-ahead mechanism is also introduced to improve tracking performance on curved paths. Lyapunov analysis proves the fixed-time stability of the proposed framework, and numerical simulations and physical experiments demonstrate that, compared to state-of-the-art adaptive LOS methods, this framework exhibits superior tracking accuracy, convergence speed, and anti-interference capability. In simulation, the time-varying look-ahead variant reduced cross-track and vertical-track RMSE by 69.37% and 67.46%, respectively, during curved-path tracking. In field experiments with an Iver 3 AUV, the proposed fixed-time guidance reduced average tracking error by 56.35% in straight-path evaluation and 27.59% in curved-path evaluation compared with conventional adaptive LOS guidance.The proposed method provides a practical guidance-level solution for achieving reliable autonomous navigation of AUVs in complex marine environments.
Read the full abstract ▾
Robotics
Beyond Dead Reckoning: A Point of View on Camera–DAS–GNSS Continuity in Road Tunnels
Lyes Saad Saoud, Hesham A. Rakha, Mona Jaber et al. (+1 more)
Road tunnels remove satellite visibility where connected and automated vehicles still require continuous, attributable, and integrity-bounded positioning. This Point of View argues that tunnel localization should be treated as infrastructure-assisted cross-modal track continuity, not as extrapolation from the last trusted satellite fix. A trusted portal satellite solution provides the global anchor, distributed acoustic sensing (DAS) continuous motion evidence, cameras sparse identity and lane anchors, onboard sensing short-term dynamics, and edge computing association, fusion, integrity monitoring, and guarded reacquisition. Ground truth is reserved for offline calibration and validation. The article develops a falsifiable research and deployment agenda for progressing from synchronized
multimodal evidence to validated, integrity-aware tunnel positioning continuity.
Read the full abstract ▾
All authors Lyes Saad Saoud, Hesham A. Rakha, Mona Jaber, Moussa Ayyash
Robotics
A Novel Robot-Assisted Learning Pedagogy for Children with ASD
Laura Boccanfuso, Erin Barney, Marilena Mademtzi et al. (+7 more)
Interaction paradigms used in
robot-assisted autism intervention have historically employed
robots as teachers, clinical assistants, or more-abled peers to promote a variety of social skills. These modalities often leverage the expertise of trained practitioners to ensure that child-
robot interactions are productive or clinically grounded to yield positive therapeutic benefits for children across the autism spectrum. Yet, despite the fact that the majority of children with autism spectrum disorder (ASD) attend mainstream schools and spend 80% or more of their time in the general classroom [27], there is a paucity of research incorporating validated classroom teaching pedagogies into
robot-assisted autism interventions. In this work, we introduce a novel teaching methodology for advancing social skills in school-aged children with ASD. We evaluate the effectiveness of a novel
robot-assisted autism intervention which incorporates the learning-by-teaching pedagogy and explores the comparative benefits of employing a
robot versus a human confederate for improved performance on a set of social skills tasks. Results show that 80% of study participants performed better in the
robot condition (mean performance in the
robot condition=63%, mean performance in the confederate condition=37%), irrespective of the scenario order. Further, 90% of all participants were significantly more engaged in the
robot condition (mean engagement:
robot=61%, confederate=32%) and, while the effect did not result in the confederate condition, analyses indicate that overall engagement in the
robot condition contributed to improved performance. These results suggest that
robots employed in a learning-by-teaching context may help enhance engagement and improve performance on a simple social skills task for children with ASD.
Read the full abstract ▾
All authors Laura Boccanfuso, Erin Barney, Marilena Mademtzi, Claire Foster, Quan Wang, Colette Torres, Lisa Chen, Brian Scassellati, Pamela Ventola, Frederick Shic
Robotics
Skill Composition for Legged Robot Reinforcement Learning
Daniel Gigliotti, Flavio Maiorana, Fabio Patrizi et al. (+1 more)
Robots, and humanoid
robots in particular, are increasingly competent at individual behaviors, each obtained by training a specialized controller. A specialized skill is quick to train, converges reliably because the problem it faces is narrow, and can be validated on its own, none of which is true of a single end-to-end policy asked to cover everything. What remains fragile is the transition between them. We argue that the composition of independent sub-policies deserves to be treated as a research problem in its own right, rather than as an implementation detail left to whatever mechanism happens to be at hand. Reliable composition is what turns a collection of separate skills into a repertoire that can be used, extended and shared. More fundamentally, if control can be passed between specialized policies safely, and at any moment, the choice of what the
robot should do next can be delegated to a component of an entirely different nature, such as a planner, an automaton or a symbolic controller, whose behavior can be inspected in advance. The policies would then only ever have to act, and what the
robot can be trusted to do would become verifiable.
Read the full abstract ▾
All authors Daniel Gigliotti, Flavio Maiorana, Fabio Patrizi, Luca Iocchi
Robotics · Cross-listed from cs.AI
Diffusion-Based Generation of Gait Trajectories
Damian Benasco, Juan Carballeira-Lopez, Jaime Ramos-Rojas et al. (+4 more)
Generation of musculoskeletal gait trajectories conditioned on patient-specific parameters remains a key challenge for wearable robotics and rehabilitation. Assistive systems such as lower-limb exoskeletons require reference trajectories that adapt to individual morphology and therapeutic goals while preserving biomechanical realism. Traditional approaches rely on hand-crafted gait templates or optimization procedures that scale poorly across subjects and walking conditions. In this work, we explore conditional
diffusion models for generating lower-limb joint-angle trajectories conditioned on gait parameters such as step length. We compare a baseline
transformer diffusion model with a controllable
diffusion transformer variant incorporating adaptive normalization and classifier-free guidance. Experiments on a
dataset of 4,590 gait cycles show that
diffusion models can generate realistic periodic gait trajectories while enabling some controllability variation in gait characteristics, highlighting their potential for personalized gait synthesis in assistive robotics.
Read the full abstract ▾
All authors Damian Benasco, Juan Carballeira-Lopez, Jaime Ramos-Rojas, Julio S. Lora-Millan, Antonio J. Del-Ama, David Rodriguez-Cianca, Pablo Lanillos
Robotics
REVOLVE: An Automated Closed-Loop Framework for Evolving Robot Manipulation with Minimal Human Intervention
Hanyu Liu, Qian Li, Yizhu Ding et al. (+8 more)
Recent advances in data-driven
robot manipulation policies have substantially improved task execution and generalization. However, real-world deployment still relies heavily on humans for failure assessment, correction, and environment reset, while models often fail to continually learn from failures and corrective experience. We present REVOLVE (
Robot Evolving via Orchestrated Loops, Verification, and Experience), an automated closed-loop framework for evolving
robot manipulation with minimal human intervention. Built on a unified software platform, REVOLVE integrates data collection, policy training and deployment, failure recovery, and continual learning into a single closed-loop workflow. Its Automated Reset and Correction (ARC) architecture automatically resets the environment and intervenes to correct policy failures. Dual-Loop
Evolution (DLE) continually improves the manipulation policy and
agent by feeding real-world interaction and failure–correction data back into policy learning and using an external mismatch
memory to refine
agent judgments. Experiments across four real-world manipulation tasks show that, after five iterations, REVOLVE improves average policy success rate by 18.5% and
agent judgment accuracy by 8.5%, while reducing human effort in data collection and deployment testing by 94.4% and 95.1%, respectively. These results demonstrate that REVOLVE transforms real-world deployment into a closed-loop learning process that continually accumulates and uses execution experience, enabling continual
evolution of both the policy and supervisory model with substantially less human intervention.
Read the full abstract ▾
All authors Hanyu Liu, Qian Li, Yizhu Ding, Jiayi Wen, Keqiang Ren, Yunsheng Ma, Tao Jian, Zhihua Wang, Zhuofan Yu, Xinran Li, Zhigong Song
Robotics
Robotic Servo Tracking of Moving Targets with Dynamic Imitation Constraints
Yazhe Luo, Sipu Ruan, Yifei Li et al. (+1 more)
Imposing explicit trajectory constraints in
robot visual servoing remains challenging. Existing tracking methods achieve fast responses by mapping visual residuals to control velocities, but they have weak constraints on the intermediate motion process, which lead to trajectory discontinuity, oscillation, or conservative behaviors. To enable constrained tracking for moving targets, this paper proposes a servo tracking method based on imitation trajectory constraints. A dynamic model describing the
robot approaching a moving target is formulated and analyzed for convergence. A time-scalable deformation mechanism and a trajectory modulation incorporating shape and amplitude components are introduced to generate a series of trajectories in real time, from which tracking points are adaptively determined to form dynamic constraints. The
robot velocity is then computed from target pose differentials or tracked key features to follow the constrained trajectory. Simulation and real-world experiments demonstrate that the proposed method can achieve dynamic obstacle avoidance and high-precision convergence compared with several state-of-the-art methods in complex environments.
Read the full abstract ▾
All authors Yazhe Luo, Sipu Ruan, Yifei Li, Diansheng Chen
Robotics
Learning Multi-Agent Task Assignment and Navigation in the Factory: from Simulation to Real Robots
Abdalwhab Bakheet Mohamed Abdalwhab, Giovanni Beltrame and David St-Onge
Reinforcement learning (RL) has shown considerable promise for robotic decision-making, yet deploying multi-
agent RL (MARL) on physical multi-
robot systems in industrial environments remains challenging. This paper investigates the real-world applicability of decentralized MARL for multi-
robot multi-machine tending. We propose Feature-fusion Multi-
Agent Proximal Policy Optimization (FMAPPO), which fuses 2D
LiDAR measurements with task-specific state information to enable safe decentralized multi-
robot task assignment and navigation. A complete simulation-to-reality pipeline was developed using high-fidelity robotic simulation and ROS2 and deployed on physical mobile-manipulator platforms operating under realistic real-world conditions, with the robotic arms disabled during the experiments. We further investigate the sensitivity of the learned policy to command update frequency, an important consideration for real-world deployment. Comparative evaluation in simulation demonstrated that FMAPPO significantly outperformed state-of-the-art baselines with a large effect size, achieving improvements of 106% and 21% in parts delivery and 48% and 11% in parts collection over MAPPO and SMAPPO, respectively. FMAPPO also increased machine utilization by 31 and 10 percentage points, respectively, while reducing collisions by 18% and 15% and increasing the safety score by 14 and 6 percentage points compared with MAPPO and SMAPPO, respectively. Furthermore, real-world experiments demonstrated that the learned decentralized policies can coordinate multiple
robots to service multiple machines while maintaining safe operation under real-world sensing and control constraints.
Videos of the real-world experiment are available online https://anonymouspapers123.github.io/FMAPPO/.
Read the full abstract ▾
Robotics
GLAM: Training a latent world model over global spatiotemporal memory for active exploration and navigation
I-Tak Ieong, Ruizhi Feng, Zhaoyang Lu et al. (+5 more)
Active exploration and semantic navigation require an embodied
agent to build
memory from partial observations, predict how the
evolution of observed spatial
memory may support future motion, and convert that prediction into actionable plans. We present GLAM, a goal-conditioned latent world model trained over global spatiotemporal
memory, and GLAM NAV, the complete navigation system built around it. Given historical map
tokens, a navigation goal, and the current
robot pose, GLAM jointly predicts future map representations and
robot-centric waypoint latents, allowing future spatial context and navigation intent to be inferred in a shared representation space. The model follows a JEPA-like latent prediction paradigm, operates directly on map-level latent
tokens rather than RGB reconstruction, and uses a pretrained waypoint encoder-decoder to supervise and decode navigation plans within GLAM NAV. Training data are collected by replaying ObjectNav expert trajectories in Habitat over HM3D v0.2 scene assets and slicing them into multi-timescale prediction samples. On a controlled HM3D-ObjectNav subset reproduction setting, GLAM NAV improves over a reproduced BSC-Nav baseline in both success rate and success weighted by path length.
Read the full abstract ▾
All authors I-Tak Ieong, Ruizhi Feng, Zhaoyang Lu, Yifei Cao, Jiayao Zhao, Leon Li, Senhua Zhu, Wenbo Ding
Robotics
Language-Grounded Semantic Target Navigation for Autonomous Surface Vehicles
Yuqing Lin and Youngroung Kim
Autonomous Surface Vehicles (ASVs) are increasingly expected to operate in ports and harbour environments, where operators may specify navigation targets through
language-based descriptions rather than predefined coordinates or fixed target identifiers. However, existing ASV navigation methods mainly execute predefined
geometric goals or task-specific objectives and give limited
attention to
language-grounded target specification. This study proposes Semantically Grounded Navigation (SGNav), a framework that enables an ASV to identify and approach a maritime target from an operator-provided description. SGNav integrates text-guided semantic grounding, harbour-aware candidate filtering, CLIP-based semantic verification, grounded target control-state construction, and Proximal Policy Optimisation-based closed-loop control. It grounds the target description in onboard RGB observations, suppresses visually or semantically irrelevant distractors, and converts the selected target into a compact control-oriented representation for policy execution. Experiments in simulated port environments show that SGNav achieves success rates of $97.0\pm1.2\%$, $92.0\pm1.5\%$, and $90.0\pm1.8\%$ across three representative target-reaching tasks, with semantic target accuracy above $97\%$ and wrong-target rates below $3\%$. SGNav also maintains $97.7$–$98.7\%$ success rates across held-out port layouts. In the Task~3 ablation study, removing harbour-aware filtering or semantic consistency reduces the success rate to $40.4\pm2.6\%$ and $50.4\pm3.1\%$, respectively. These findings demonstrate the importance of semantic grounding, harbour-aware filtering, and semantic verification for reliable
language-grounded ASV navigation. These results indicate that the proposed perception-to-control framework can support
language-grounded target approach manoeuvres of ASV under the complex port environments.
Read the full abstract ▾
Robotics
NavPatch: Evidence-Guided Object-Level Costmap Correction with Vision-Language Models
Shiji Sun, Xingyu Tao, Hao Wang et al. (+2 more)
Mobile
robots typically rely on
geometric maps for obstacle avoidance and path planning, but the resulting obstacle representation does not always match how an object should affect navigation. A low lying cable may be missed, a flexible curtain may create spurious blockage, and a traffic cone may require an exclusion region larger than its observed footprint. We present NavPatch, an object level correction layer that assigns ADD, REMOVE, or EXTEND to navigation relevant object categories through periodic scene understanding with a
vision-
language model. Open vocabulary grounding localizes object instances, and
LiDAR and RGB-D observations provide 3D support. Observation quality filtering and cross frame maintenance determine when each correction patch is committed, replaced, or revoked. In 50 real
robot trials across five layouts, NavPatch achieves an overall success rate of 86.0%. An ablation study of four configurations with 200 runs in total shows that NavPatch improves the success rate from 70.0% to 86.0% and reduces the false commit rate from 68.4% to 40.7% compared with updates based only on the current observation.
Read the full abstract ▾
All authors Shiji Sun, Xingyu Tao, Hao Wang, Ling Wang, Zhengyi Chen
Robotics
Fault Diagnosis for Underwater Vehicles using Moving Horizon Estimation and Gaussian Processes
Fotis Panetsos and Kostas J. Kyriakopoulos
This work proposes a model-based fault detection and diagnosis framework for underwater vehicles subject to actuator faults that explicitly accounts for the presence of unmodeled dynamics. To this end, a Moving Horizon
Estimator (MHE) is developed to estimate the lumped disturbance, capturing both unmodeled and fault effects. Gaussian Processes (GPs) are employed to approximate the unmodeled dynamics, providing predictions of the corresponding mean and uncertainty across diverse operating conditions. During online operation, the residual between the MHE lumped disturbance estimate and the GP prediction is evaluated using a Generalized Likelihood Ratio Test. By incorporating GP-based predictions within the diagnostic framework, robustness to unmodeled dynamics is achieved, enabling effective fault detection and isolation as well as accurate quantitative estimation of fault magnitude. The proposed methodology is experimentally validated in a laboratory water tank, demonstrating reliable diagnostic performance under both open-loop and closed-loop control.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
DynEoMT: Learning Object Dynamicity from Online Segmentation Queries
Calvin Galagain, Martyna Poreba and François Goulette
Video segmentation models recognize and track objects over time, but they do not indicate whether each segmented region moves independently of the observing camera. This dynamicity attribute cannot be inferred from semantics alone and is confounded by camera ego-motion. We introduce \method, an online framework that augments query-based
video segmentation with region-level dynamicity prediction. It jointly produces the original segmentation outputs and a dynamic or static state for each predicted region. At inference, DynEoMT uses only the current frame and propagated queries, without optical flow, depth, camera pose, previous RGB frames, or feature maps. Because established
video segmentation
benchmarks do not annotate this attribute, we also introduce a class-agnostic offline supervision pipeline using camera-compensated optical flow and confidence-aware temporal filtering. Across VIPSeg, OVIS, YouTube-VIS 2022, and VSPW, DynEoMT achieves balanced accuracies of 84.3, 68.0, 68.6, and 87.6, respectively, while largely preserving segmentation performance. These results show that segmentation-region dynamicity can be learned from propagated queries, enabling its online prediction without a dedicated motion-processing pipeline at inference. The complete code will be released as open source to enable full reproduction of the method and experiments.
Read the full abstract ▾
Robotics
EcoBoat: Design and Experimental Validation of an Autonomous Body-Board Boat For Cleaning Water Bodies
M. Aman Ansari, Saifullah Khan, Rahul Kulkarni et al. (+1 more)
Cleaning water bodies such as swimming pools and lakes typically demands significant manual effort or reliance on costly, sensor-intensive robotic systems. This paper presents EcoBoat, a low-cost autonomous surface vehicle built on a modified hull, designed to collect floating debris in both indoor and outdoor water bodies. For indoor environments, EcoBoat uses ultrasonic sensors to detect boundaries and obstacles, combining random-walk motion with boundary-following behavior. For outdoor environments, it relies on GPS-based geofencing paired with a random-walk strategy for area coverage. A key design feature is a Tesla-valve-inspired collection basket that allows debris intake during forward motion while preventing its escape during turning or braking. Field experiments in pools and lakes validated the design through iterative refinement, demonstrating that a minimalist sensing and computation approach can achieve effective, versatile debris collection across diverse water bodies.
Read the full abstract ▾
All authors M. Aman Ansari, Saifullah Khan, Rahul Kulkarni, PB Sujit
Robotics
EMoG: Emotion-Modulated Gait Generation for Expressive Humanoid Locomotion
Yi Lu, Tianhao Jiang, Honglong Tian et al. (+6 more)
Existing humanoid locomotion systems primarily focus on stability and task execution, while integrating expressiveness with explicit locomotion control remains challenging. We propose EMoG, an emotion-modulated gait generation framework for expressive humanoid locomotion. EMoG introduces an emotional-style code with continuously adjustable intensity. Conditioned on this code and physical commands, a lightweight MLP generates expressive, command-consistent periodic gait trajectories in real time, which are tracked by a unified reinforcement learning policy for physical execution. To support training, we collect a large-scale emotion-annotated gait
dataset from professional performers and develop an automated pipeline to extract physically consistent periodic gait cycles. EMoG also integrates an LLM-based parser that converts free-form
language into emotional style and motion parameters for interactive control. Experiments demonstrate that our system achieves continuous gait-style modulation with perceptible expressive cues while maintaining command tracking. EMoG provides a practical approach to parameterized emotional-style walking for human-
robot interaction.
Read the full abstract ▾
All authors Yi Lu, Tianhao Jiang, Honglong Tian, Yumeng Zhang, Qingrui Zhao, Zhengtao Wang, Xiao-Xiao Long, Qiu Shen, Xun Cao
Robotics
Learning-Based Dynamic Obstacle Avoidance for a UAV Using Only Three Range Sensors
Mohammad Reza Ranjbar Divkoti and A. Pedro Aguiar
We present a learning-based approach to kinodynamic online motion planning for an Unmanned Aerial Vehicle (
UAV) operating at a fixed altitude in unknown dynamic environments, where real-time avoidance of both static and dynamic obstacles must be achieved under conditions of extreme partial observability. The
UAV is controlled with a single degree of freedom (yaw only), resulting in constrained, nonholonomic motion similar to fixed-wing platforms. The proposed framework integrates a behavior grid map representation with Deep Reinforcement Learning (DRL), using Proximal Policy Optimization (PPO) for stable policy learning in continuous control. The key idea is the co-design of a state representation and control policy that enables reliable navigation using only three low-cost directional range sensors, without reliance on dense sensing modalities such as
LiDAR or
vision-based systems. The behavior grid map dynamically aggregates sparse measurements into a structured local representation that supports real-time decision-making for obstacle avoidance and target reaching. Extensive simulations across environments of varying sizes and obstacle densities demonstrate that the proposed standard and enhanced methods achieve higher success rates than PPO variants and Model Predictive Control (MPC) (94% vs. 79–90% in small-scale high-congestion scenarios, and 83% vs. 62–71% in large-scale high-congestion scenarios), while maintaining real-time performance. Real-world experiments across four scenarios further confirm practical feasibility, with consistent target-reaching behaviour and no collisions under the tested conditions.
Read the full abstract ▾