Robotics
FAST-EQA: Efficient Embodied Question Answering with Global and Local Region Relevancy
Haochen Zhang, Nirav Savaliya, Faizan Siddiqui et al. (+1 more)
Embodied Question Answering (EQA) combines visual scene understanding, goal-directed exploration, spatial and temporal
reasoning under partial observability. A central challenge is to confine physical search to question-relevant subspaces while maintaining a compact, actionable
memory of observations. Furthermore, for real-world deployment, fast inference time during exploration is crucial. We introduce FAST-EQA, a question-conditioned framework that (i) identifies likely visual targets, (ii) scores global regions of interest to guide
navigation, and (iii) employs Chain-of-Thought (CoT)
reasoning over visual
memory to answer confidently. FAST-EQA maintains a bounded scene
memory that stores a fixed-capacity set of region-target hypotheses and updates them online, enabling robust handling of both single and multi-target questions without unbounded growth. To expand coverage efficiently, a global exploration policy treats narrow openings and doors as high-value frontiers, complementing local target seeking with minimal computation. Together, these components focus the
agent's
attention, improve scene coverage, and improve answer reliability while running substantially faster than prior approaches. On HMEQA and EXPRESS-Bench, FAST-EQA achieves state-of-the-art performance, while performing competitively on OpenEQA and MT-HM3D.
Read the full abstract ▾
All authors Haochen Zhang, Nirav Savaliya, Faizan Siddiqui, Enna Sachdeva
Robotics
Robot-Assisted Social Dining as a White Glove Service
Atharva S Kashyap, Ugne Aleksandra Morkute and Patricia Alves-Oliveira
Robot-assisted feeding enables people with disabilities who require assistance eating to enjoy a meal independently and with dignity. However, existing systems have only been tested in-lab or in-home, leaving in-the-wild social dining contexts (e.g., restaurants) largely unexplored. Designing a
robot for such contexts presents unique challenges, such as dynamic and unsupervised dining environments that a
robot needs to account for and respond to. Through speculative participatory design with people with disabilities, supported by semi-structured interviews and a custom AI-based visual storyboarding tool, we uncovered ideal scenarios for in-the-wild social dining. Our key insight suggests that such systems should: embody the principles of a white glove service where the
robot (1) supports
multimodal inputs and unobtrusive outputs; (2) has contextually sensitive social behavior and prioritizes the user; (3) has expanded roles beyond feeding; (4) adapts to other relationships at the dining table. Our work has implications for in-the-wild and group contexts of
robot-assisted feeding.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
RaCo: Ranking and Covariance for Practical Learned Keypoints
Abhiram Shenoi, Philipp Lindenberger, Paul-Edouard Sarlin et al. (+1 more)
This paper introduces RaCo, a lightweight neural network designed to learn robust and versatile keypoints suitable for a variety of 3D computer
vision tasks. The model integrates three key components: the repeatable keypoint detector, a differentiable ranker to maximize matches with a limited number of keypoints, and a covariance
estimator to quantify spatial uncertainty in metric scale. Trained on perspective image crops only, RaCo operates without the need for covisible image pairs. It achieves strong rotational robustness through extensive data augmentation, even without the use of computationally expensive equivariant network architectures. The method is evaluated on several challenging
datasets, where it demonstrates state-of-the-art performance in keypoint repeatability and two-view matching, particularly under large in-plane rotations. Ultimately, RaCo provides an effective and simple strategy to independently estimate keypoint ranking and metric covariance without additional labels, detecting interpretable and repeatable interest points. The code is available at https://github.com/cvg/RaCo.
Read the full abstract ▾
All authors Abhiram Shenoi, Philipp Lindenberger, Paul-Edouard Sarlin, Marc Pollefeys
Robotics
MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction
Qiang Zhang, Jiahao Ma, Peiran Liu et al. (+20 more)
Humanoid motion control has witnessed significant breakthroughs in recent years, with deep reinforcement learning (RL) emerging as a primary catalyst for achieving complex, human-like behaviors. However, the high dimensionality and intricate dynamics of
humanoid robots make manual motion design impractical, leading to a heavy reliance on expensive motion capture (MoCap) data. These
datasets are not only costly to acquire but also frequently lack the necessary
geometric context of the surrounding physical environment. Consequently, existing motion synthesis frameworks often suffer from a decoupling of motion and scene, resulting in physical inconsistencies such as contact slippage or mesh penetration during terrain-aware tasks. In this work, we present MeshMimic, an innovative framework that bridges 3D scene reconstruction and
embodied intelligence to enable
humanoid robots to learn coupled "motion-terrain" interactions directly from
video. By leveraging state-of-the-art 3D
vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects. We introduce an optimization
algorithm based on kinematic consistency to extract high-quality motion data from noisy visual reconstructions, alongside a contact-invariant retargeting method that transfers human-environment interaction features to the
humanoid agent. Experimental results demonstrate that MeshMimic achieves robust, highly dynamic performance across diverse and challenging terrains. Our approach proves that a low-cost pipeline utilizing only consumer-grade
monocular sensors can facilitate the training of complex physical interactions, offering a scalable path toward the autonomous
evolution of
humanoid robots in unstructured environments.
Read the full abstract ▾
All authors Qiang Zhang, Jiahao Ma, Peiran Liu, Shuai Shi, Zeran Su, Zifan Wang, Jingkai Sun, Wei Cui, Jialin Yu, Gang Han, Wen Zhao, Pihai Sun, Kangning Yin, Jiaxu Wang, Jiahang Cao, Lingfeng Zhang, Hao Cheng, Xiaoshuai Hao, Yiding Ji, Junwei Liang, Jian Tang, Renjing Xu, Yijie Guo
Robotics
Lifelong Scalable Multi-Agent Realistic Testbed and A Comprehensive Study on Design Choices in Lifelong AGV Fleet Management Systems
Jingtian Yan, Yulun Zhang, Zhenting Liu et al. (+5 more)
We present Lifelong Scalable Multi-
Agent Realistic Testbed (LSMART), an open-source simulator to evaluate any Multi-
Agent Path Finding (MAPF)
algorithm in a Fleet Management System (FMS) with Automated Guided Vehicles (AGVs). MAPF aims to move a group of
agents from their corresponding starting locations to their goals. Lifelong MAPF (LMAPF) is a variant of MAPF that continuously assigns new goals for
agents to reach. LMAPF applications, such as autonomous warehouses, often require a centralized, lifelong system to coordinate the movement of a fleet of
robots, typically AGVs. However, existing works on MAPF and LMAPF often assume simplified kinodynamic models, such as pebble motion, as well as perfect execution and communication for AGVs. Prior work has presented SMART, a software capable of evaluating any MAPF
algorithms while considering
agent kinodynamics, communication delays, and execution uncertainties. However, SMART is designed for MAPF, not LMAPF. Generalizing SMART to an FMS requires many more design choices. First, an FMS parallelizes planning and execution, raising the question of when to plan. Second, given planners with varying optimality and differing
agent-model assumptions, one must decide how to plan. Third, when the planner fails to return valid solutions, the system must determine how to recover. In this paper, we first present LSMART, an open-source simulator that incorporates all these considerations to evaluate any MAPF
algorithms in an FMS. We then provide experiment results based on state-of-the-art methods for each design choice, offering guidance on how to effectively design centralized lifelong AGV Fleet Management Systems. LSMART is available at https://smart-mapf.github.io/lifelong-smart.
Read the full abstract ▾
All authors Jingtian Yan, Yulun Zhang, Zhenting Liu, Han Zhang, He Jiang, Jingkai Chen, Stephen F. Smith, Jiaoyang Li
Robotics
Estimating Human Muscular Fatigue in Dynamic Collaborative Robotic Tasks with Learning-Based Models
Feras Kiki, Pouya P. Niaz, Alireza Madani et al. (+1 more)
Assessing human muscle fatigue is critical for optimizing performance and safety in physical human-
robot interaction(pHRI). This work presents a data-driven framework to estimate fatigue in dynamic, cyclic pHRI using arm-mounted surface electromyography(sEMG). Subject-specific machine-learning regression models(Random Forest, XGBoost, and Linear Regression predict the fraction of cycles to fatigue(FCF) from three frequency-domain and one time-domain EMG features, and are benchmarked against a convolutional neural network(CNN) that ingests spectrograms of filtered EMG. Framing fatigue estimation as regression (rather than classification) captures continuous progression toward fatigue, supporting earlier detection, timely intervention, and adaptive
robot control. In experiments with ten participants, a collaborative
robot under admittance control guided repetitive lateral (left-right) end-effector motions until muscular fatigue. Average FCF RMSE across participants was 20.8+/-4.3% for the CNN, 23.3+/-3.8% for Random Forest, 24.8+/-4.5% for XGBoost, and 26.9+/-6.1% for Linear Regression. To probe cross-task generalization, one participant additionally performed unseen vertical (up-down) and circular repetitions; models trained only on lateral data were tested directly and largely retained accuracy, indicating robustness to changes in movement direction, arm kinematics, and muscle recruitment, while Linear Regression deteriorated. Overall, the study shows that both feature-based ML and spectrogram-based DL can estimate remaining work capacity during repetitive pHRI, with the CNN delivering the lowest error and the tree-based models close behind. The reported transfer to new motion patterns suggests potential for practical fatigue monitoring without retraining for every task, improving operator protection and enabling fatigue-aware shared autonomy, for safer fatigue-adaptive pHRI control.
Read the full abstract ▾
All authors Feras Kiki, Pouya P. Niaz, Alireza Madani, Cagatay Basdogan
Robotics
Spatially-Aware Adaptive Trajectory Optimization with Controller-Guided Feedback for Autonomous Racing
Alexander Wachter, Alexander Willert, Marc-Philip Ecker et al. (+1 more)
We present a closed-loop framework for autonomous raceline optimization that combines NURBS-based trajectory representation, CMA-ES global trajectory optimization, and controller-guided spatial feedback. Instead of treating tracking errors as transient disturbances, our method exploits them as informative signals of local track characteristics via a Kalman-inspired spatial update. This enables the construction of an adaptive, acceleration-based constraint map that iteratively refines trajectories toward near-optimal performance under spatially varying track and vehicle behavior. In simulation, our approach achieves a 17.38% lap time reduction compared to a controller parametrized with maximum static acceleration. On real hardware, tested with different tire compounds ranging from high to low friction, we obtain a 7.60% lap time improvement without explicitly parametrizing friction. This demonstrates robustness to changing grip conditions in real-world scenarios.
Read the full abstract ▾
All authors Alexander Wachter, Alexander Willert, Marc-Philip Ecker, Christian Hartl-Nesic
Robotics
SpecFuse: A Spectral-Temporal Fusion Predictive Control Framework for UAV Landing on Oscillating Marine Platforms
Haichao Liu, Yufeng Hu, Shuang Wang et al. (+3 more)
Autonomous landing of Uncrewed Aerial Vehicles (
UAVs) on oscillating marine platforms is severely constrained by wave-induced multi-frequency oscillations, wind disturbances, and prediction phase lags in motion prediction. Existing methods either treat platform motion as a general random process or lack explicit modeling of wave spectral characteristics, leading to suboptimal performance under dynamic sea conditions. To address these limitations, we propose SpecFuse: a novel spectral-temporal fusion predictive control framework that integrates frequency-domain wave decomposition with time-domain recursive state estimation for high-precision 6-DoF motion
forecasting of Uncrewed Surface Vehicles (USVs). The framework explicitly models dominant wave harmonics to mitigate phase lags, refining predictions in real time via IMU data without relying on complex calibration. Additionally, we design a hierarchical control architecture featuring a sampling-based HPO-RRT*
algorithm for dynamic trajectory planning under non-convex constraints and a learning-augmented predictive controller that fuses data-driven disturbance compensation with optimization-based execution. Extensive validations (2,000 simulations + 8 lake experiments) show our approach achieves a 3.2 cm prediction error, 4.46 cm landing deviation, 98.7% / 87.5% success rates (simulation / real-world), and 82 ms latency on embedded hardware, outperforming state-of-the-art methods by 44%-48% in accuracy. Its robustness to wave-wind coupling disturbances supports critical maritime missions such as search and rescue and environmental monitoring. All code, experimental configurations, and
datasets will be released as open-source to facilitate reproducibility.
Read the full abstract ▾
All authors Haichao Liu, Yufeng Hu, Shuang Wang, Kangjun Guo, Jun Ma, Jinni Zhou
Robotics
Grip as Needed, Glide on Demand: Ultrasonic Lubrication for Robotic Locomotion
Mostafa A. Atalla, Daan van Bemmel, Jack Cummings et al. (+3 more)
Friction is the essential mediator of terrestrial locomotion, yet in robotic systems it is almost always treated as a passive property fixed by surface materials and conditions. Here, we introduce ultrasonic lubrication as a method to actively control friction in robotic locomotion. By exciting resonant structures at ultrasonic frequencies, contact interfaces can dynamically switch between "grip" and "slip" states, enabling locomotion. We developed two friction control modules, a cylindrical design for lumen-like environments and a flat-plate design for external surfaces, and integrated them into bio-inspired systems modeled after inchworm and wasp ovipositor locomotion. Both systems achieved bidirectional locomotion with nearly perfect locomotion efficiencies that exceeded 90%. Friction characterization experiments further demonstrated substantial friction reduction across various surfaces, including rigid, soft, granular, and biological tissue interfaces, under dry and wet conditions, and on surfaces with different levels of roughness, confirming the broad applicability of ultrasonic lubrication to locomotion tasks. These findings establish ultrasonic lubrication as a viable active friction control mechanism for robotic locomotion, with the potential to reduce design complexity and improve efficiency of robotic locomotion systems.
Read the full abstract ▾
All authors Mostafa A. Atalla, Daan van Bemmel, Jack Cummings, Paul Breedveld, Michaël Wiertlewski, Aimée Sakes
Robotics
Constraining Streaming Flow Models for Adapting Learned Robot Trajectory Distributions
Jieting Long, Dechuan Liu, Weidong Cai et al. (+2 more)
Robot motion distributions often exhibit multi-modality and require flexible generative models for accurate representation. Streaming Flow Policies (SFPs) have recently emerged as a powerful paradigm for generating
robot trajectories by integrating learned velocity fields directly in action space, enabling smooth and reactive control. However, existing formulations lack mechanisms for adapting trajectories post-training to enforce safety and task-specific constraints. We propose Constraint-Aware Streaming Flow (CASF), a framework that augments streaming flow policies with constraint-dependent metrics that reshape the learned velocity field during execution. CASF models each constraint, defined in either the
robot's workspace or configuration space, as a differentiable distance function that is converted into a local metric and pulled back into the
robot's control space. Far from restricted regions, the resulting metric reduces to the identity; near constraint boundaries, it smoothly attenuates or redirects motion, effectively deforming the underlying flow to maintain safety. This allows trajectories to be adapted in real time, ensuring that
robot actions respect joint limits, avoid collisions, and remain within feasible workspaces, while preserving the multi-modal and reactive properties of streaming flow policies. We demonstrate CASF in simulated and real-world manipulation tasks, showing that it produces constraint-satisfying trajectories that remain smooth, feasible, and dynamically consistent, outperforming standard post-hoc projection baselines.
Read the full abstract ▾
All authors Jieting Long, Dechuan Liu, Weidong Cai, Ian Manchester, Weiming Zhi
Robotics
VLM-DEWM: Dynamic External World Model for Verifiable and Resilient Vision-Language Planning in Manufacturing
Guoqin Tang, Qingxuan Jia, Gang Chen et al. (+4 more)
Vision-
language model (VLM) shows promise for high-level planning in smart manufacturing, yet their deployment in dynamic workcells faces two critical challenges: (1) stateless operation, they cannot persistently track out-of-view states, causing world-state drift; and (2) opaque
reasoning, failures are difficult to diagnose, leading to costly blind retries. This paper presents VLM-DEWM, a cognitive architecture that decouples VLM
reasoning from world-state management through a persistent, queryable Dynamic External World Model (DEWM). Each VLM decision is structured into an Externalizable
Reasoning Trace (ERT), comprising action proposal, world belief, and causal assumption, which is validated against DEWM before execution. When failures occur, discrepancy analysis between predicted and observed states enables targeted recovery instead of global replanning. We evaluate VLM-DEWM on multi-station assembly, large-scale facility exploration, and real-
robot recovery under induced failures. Compared to baseline
memory-augmented VLM systems, VLM DEWM improves state-tracking accuracy from 56% to 93%, increases recovery success rate from below 5% to 95%, and significantly reduces computational overhead through structured
memory. These results establish VLM-DEWM as a verifiable and resilient solution for long-horizon robotic operations in dynamic manufacturing environments.
Read the full abstract ▾
All authors Guoqin Tang, Qingxuan Jia, Gang Chen, Tong Li, Zeyuan Huang, Zihang Lv, Ning Ji
Robotics
Young-Chae Son, Jung-Woo Lee, Yoon-Ji Choi et al. (+2 more)
In robotics,
Vision-
Language-Action (VLA) models that integrate diverse
multimodal signals from multi-view inputs have emerged as an effective approach. However, most prior work adopts static fusion that processes all visual inputs uniformly, which incurs unnecessary computational overhead and allows task-irrelevant background information to act as noise. Inspired by the principles of human active perception, we propose a dynamic information fusion framework designed to maximize the efficiency and robustness of VLA models. Our approach introduces a lightweight adaptive routing architecture that analyzes the current text
prompt and observations from a wrist-mounted camera in real-time to predict the task-relevance of multiple camera views. By conditionally attenuating computations for views with low informational utility and selectively providing only essential visual features to the policy network, Our framework achieves computation efficiency proportional to task relevance. Furthermore, to efficiently secure large-scale annotation data for router training, we established an automated labeling pipeline utilizing
Vision-
Language Models (VLMs) to minimize data collection and annotation costs. Experimental results in real-world robotic manipulation scenarios demonstrate that the proposed approach achieves significant improvements in both inference efficiency and control performance compared to existing VLA models, validating the effectiveness and practicality of dynamic information fusion in resource-constrained, real-time
robot control environments.
Read the full abstract ▾
All authors Young-Chae Son, Jung-Woo Lee, Yoon-Ji Choi, Dae-Kwan Ko, Soo-Chul Lim
Robotics
Efficient Knowledge Transfer for Jump-Starting Control Policy Learning of Multirotors through Physics-Aware Neural Architectures
Welf Rehberg, Mihir Kulkarni, Philipp Weiss et al. (+1 more)
Efficiently training control policies for
robots is a major challenge that can greatly benefit from utilizing knowledge gained from training similar systems through cross-embodiment knowledge transfer. In this work, we focus on accelerating policy training using a library-based initialization scheme that enables effective knowledge transfer across multirotor configurations. By leveraging a physics-aware neural control architecture that combines a reinforcement learning-based controller and a supervised control allocation network, we enable the reuse of previously trained policies. To this end, we utilize a policy evaluation-based similarity measure that identifies suitable policies for initialization from a library. We demonstrate that this measure correlates with the reduction in environment interactions needed to reach target performance and is therefore suited for initialization. Extensive simulation and real-world experiments confirm that our control architecture achieves state-of-the-art control performance, and that our initialization scheme saves on average up to $73.5\%$ of environment interactions (compared to training a policy from scratch) across diverse quadrotor and hexarotor designs, paving the way for efficient cross-embodiment transfer in reinforcement learning.
Read the full abstract ▾
All authors Welf Rehberg, Mihir Kulkarni, Philipp Weiss, Kostas Alexis
Robotics
HIMM: Human-Inspired Long-Term Memory Modeling for Embodied Exploration and Question Answering
Ji Li, Bo Wang, Jing Xia et al. (+2 more)
Deploying
Multimodal Large
Language Models as the
brain of
embodied agents remains challenging, particularly under long-horizon observations and limited context budgets. Existing
memory assisted methods often rely on textual summaries, which discard rich visual and spatial details and remain brittle in non-stationary environments. In this work, we propose a non-parametric
memory framework that explicitly disentangles episodic and semantic
memory for
embodied exploration and question answering. Our
retrieval-first,
reasoning-assisted paradigm recalls episodic experiences via semantic similarity and verifies them through visual
reasoning, enabling robust reuse of past observations without rigid
geometric alignment. In parallel, we introduce a program-style rule extraction mechanism that converts experiences into structured, reusable semantic
memory, facilitating cross-environment generalization. Extensive experiments demonstrate state-of-the-art performance on
embodied question answering and exploration
benchmarks, yielding a 7.3% gain in LLM-Match and an 11.4% gain in LLM MatchXSPL on A-EQA, as well as +7.7% success rate and +6.8% SPL on GOAT-Bench. Analyses reveal that our episodic
memory primarily improves exploration efficiency, while semantic
memory strengthens complex
reasoning of
embodied agents.
Read the full abstract ▾
All authors Ji Li, Bo Wang, Jing Xia, Mingyi Li, Shiyan Hu
Robotics
Lyapunov-Based PI-Like Control for Robust Trajectory Tracking of a Four-Wheel Independently Driven and Steered Robot: Design and Experimental Validation
Branimir Ćaran, Vladimir Milić, Marko Švaco et al. (+1 more)
In this paper, a Lyapunov-based synthesis of a PI-like controller is proposed for robust trajectory tracking of an independently driven and steered four-wheel mobile
robot. For the
robot considered in this work, an explicit structurally verified mathematical model is used to enable systematic controller design with rigorous stability guarantees suitable for real time implementation. An augmented Lyapunov-based practical stability analysis is developed for the combined velocity-error and integral-error dynamics of the inner loop, yielding explicit bounds and sufficient conditions for practical stability and uniform ultimate boundedness of the combined velocity-error and integral-error state. The resulting control law retains a PI-like structure with model-based feedforward compensation, making it suitable for implementation on standard embedded platforms while improving robustness against configuration dependent residual dynamics and unmodelled effects. The effectiveness and robustness of the proposed design are demonstrated experimentally on a four-wheel independently steered and independently driven mobile
robot platform, under both horizontal and vertical operating conditions and benchmarked against a PI controller and a sliding-mode controller.
Read the full abstract ▾
All authors Branimir Ćaran, Vladimir Milić, Marko Švaco, Bojan Jerbić
Robotics
One Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representation
Zerui Li, Hongpei Zheng, Fangguo Zhao et al. (+5 more)
A navigable
agent needs to understand both high-level semantic instructions and precise spatial perceptions. Building
navigation agents centered on
Multimodal Large
Language Models (MLLMs) demonstrates a promising solution due to their powerful generalization ability. However, the current tightly coupled design dramatically limits system performance. In this work, we propose a decoupled design that separates low-level spatial state estimation from high-level semantic planning. Unlike previous methods that rely on predefined, oversimplified textual maps, we introduce an interactive metric world representation that maintains rich and consistent information, allowing MLLMs to interact with and reason on it for decision-making. Furthermore, counterfactual
reasoning is introduced to further elicit MLLMs' capacity, while the metric world representation ensures the physical validity of the produced actions. We conduct comprehensive experiments in both simulated and real-world environments. Our method establishes a new zero-shot state-of-the-art, achieving 48.8% Success Rate (SR) in R2R-CE and 42.2% in RxR-CE
benchmarks. Furthermore, to validate the versatility of our metric representation, we demonstrate zero-shot sim-to-real transfer across diverse embodiments, including a wheeled TurtleBot 4 and a custom-built aerial drone. These real-world deployments verify that our decoupled framework serves as a robust, domain-invariant interface for
embodied Vision-and-
Language navigation.
Read the full abstract ▾
All authors Zerui Li, Hongpei Zheng, Fangguo Zhao, Aidan Chan, Jian Zhou, Sihao Lin, Shijie Li, Qi Wu
Robotics
Hybrid F' and ROS2 Architecture for Vision-Based Autonomous Flight: Design and Experimental Validation
Abdelrahman Metwally, Monijesu James, Aleksey Fedoseev et al. (+3 more)
Autonomous aerospace systems require architectures that balance deterministic real-time control with advanced perception capabilities. This paper presents an integrated system combining NASA's F' flight software framework with ROS2 middleware via Protocol Buffers bridging. We evaluate the architecture through a 32.25-minute indoor quadrotor flight test using
vision-based
navigation. The
vision system achieved 87.19 Hz position estimation with 99.90% data continuity and 11.47 ms mean latency, validating real-time performance requirements. All 15 ground commands executed successfully with 100 % success rate, demonstrating robust F'–PX4 integration. System resource utilization remained low (15.19 % CPU, 1,244 MB RAM) with zero stale telemetry messages, confirming efficient operation on embedded platforms. Results validate the feasibility of hybrid flight-software architectures combining certification-grade determinism with flexible autonomy for autonomous aerial vehicles.
Read the full abstract ▾
All authors Abdelrahman Metwally, Monijesu James, Aleksey Fedoseev, Miguel Altamirano Cabrera, Dzmitry Tsetserukou, Andrey Somov
Robotics
ActionCodec: What Makes for Good Action Tokenizers
Zibin Dong, Yicheng Liu, Shiduo Zhang et al. (+8 more)
Vision-
Language-Action (VLA) models leveraging the native autoregressive paradigm of
Vision-
Language Models (VLMs) have demonstrated superior instruction-following and training efficiency. Central to this paradigm is action tokenization, yet its design has primarily focused on reconstruction fidelity, failing to address its direct impact on VLA optimization. Consequently, the fundamental question of what makes for good action tokenizers remains unanswered. In this paper, we bridge this gap by establishing design principles specifically from the perspective of VLA optimization. We identify a set of best practices based on information-theoretic insights, including maximized temporal
token overlap, minimized vocabulary redundancy, enhanced
multimodal mutual information, and
token independence. Guided by these principles, we introduce ActionCodec, a high-performance action tokenizer that significantly enhances both training efficiency and VLA performance across diverse simulation and real-world
benchmarks. Notably, on LIBERO, a SmolVLM2-2.2B fine-tuned with ActionCodec achieves a 95.5% success rate without any robotics pre-training. With advanced architectural enhancements, this reaches 97.4%, representing a new SOTA for VLA models without robotics pre-training. We believe our established design principles, alongside the released model, will provide a clear roadmap for the community to develop more effective action tokenizers.
Read the full abstract ▾
All authors Zibin Dong, Yicheng Liu, Shiduo Zhang, Baijun Ye, Yifu Yuan, Fei Ni, Jingjing Gong, Xipeng Qiu, Hang Zhao, Yinchuan Li, Jianye Hao
Robotics
Fluoroscopy-Constrained Magnetic Robot Control via Zernike-Based Field Modeling and Nonlinear MPC
Xinhao Chen, Hongkun Yao, Anuruddha Bhattacharjee et al. (+4 more)
Magnetic actuation enables
surgical robots to navigate complex anatomical pathways while reducing tissue trauma and improving
surgical precision. However, clinical deployment is limited by the challenges of controlling such systems under fluoroscopic imaging, which provides low frame rate and noisy pose feedback. This paper presents a control framework that remains accurate and stable under such conditions by combining a nonlinear model predictive control (NMPC) framework that directly outputs coil currents, an analytically differentiable
magnetic field model based on Zernike
polynomials, and a Kalman filter to estimate the
robot state. Experimental validation is conducted with two
magnetic robots in a 3D-printed fluid workspace and a spine phantom replicating drug delivery in the epidural space. Results show the proposed control method remains highly accurate when feedback is downsampled to 3 Hz with added Gaussian noise (sigma = 2 mm), mimicking clinical fluoroscopy. In the spine phantom experiments, the proposed method successfully executed a drug delivery trajectory with a root mean square (RMS) position error of 1.18 mm while maintaining safe clearance from critical anatomical boundaries.
Read the full abstract ▾
All authors Xinhao Chen, Hongkun Yao, Anuruddha Bhattacharjee, Suraj Raval, Lamar O. Mair, Yancy Diaz-Mercado, Axel Krieger
Robotics
A Comparison of Bayesian Prediction Techniques for Mobile Robot Trajectory Tracking
Jose Luis Peralta-Cabezas, Miguel Torres-Torriti and Marcelo Guarini-Hermann
This paper presents a performance comparison of different estimation and prediction techniques applied to the problem of tracking multiple
robots. The main performance criteria are the magnitude of the estimation or prediction error, the computational effort and the robustness of each method to non-Gaussian noise. Among the different techniques compared are the well known Kalman filters and their different variants (e.g. extended and unscented), and the more recent techniques relying on Sequential Monte Carlo Sampling methods, such as particle filters and Gaussian Mixture Sigma Point Particle Filter.
Read the full abstract ▾
Robotics
Feasibility-aware Imitation Learning from Observation with Multimodal Feedback
Kei Takahashi, Hikaru Sasaki and Takamitsu Matsubara
Imitation learning frameworks that learn
robot control policies from demonstrators' motions via hand-mounted demonstration interfaces have attracted increasing
attention. However, due to differences in physical characteristics between demonstrators and
robots, this approach faces two limitations: i) the demonstration data do not include
robot actions, and ii) the demonstrated motions may be infeasible for
robots. These limitations make policy learning difficult. To address them, we propose Feasibility-Aware Behavior Cloning from Observation (FABCO). FABCO integrates behavior cloning from observation, which complements
robot actions using
robot dynamics models, with feasibility estimation. In feasibility estimation, the demonstrated motions are evaluated using a
robot-dynamics model, learned from the
robot's execution data, to assess reproducibility under the
robot's dynamics. The estimated feasibility is used for
multimodal feedback and feasibility-aware policy learning to improve the demonstrator's motions and learn robust policies.
Multimodal feedback provides feasibility through the demonstrator's visual and haptic senses to promote feasible demonstrated motions. Feasibility-aware policy learning reduces the influence of demonstrated motions that are infeasible for
robots, enabling the learning of policies that
robots can execute stably. We conducted experiments with 15 participants on two tasks and confirmed that FABCO improves imitation learning performance by more than 3.2 times compared to the case without feasibility feedback.
Read the full abstract ▾
Robotics
OSCAR: An Ovipositor-Inspired Self-Propelling Capsule Robot for Colonoscopy
Mostafa A. Atalla, Anand S. Sekar, Remi van Starkenburg et al. (+4 more)
Self-propelling robotic capsules eliminate shaft looping of conventional colonoscopy, reducing patient discomfort. However, reliably moving within the slippery, viscoelastic environment of the colon remains a significant challenge. We present OSCAR, an ovipositor-inspired self-propelling capsule
robot that translates the transport strategy of parasitic wasps into a propulsion mechanism for colonoscopy. OSCAR mechanically encodes the ovipositor-inspired motion pattern through a spring-loaded cam system that drives twelve circumferential sliders in a coordinated, phase-shifted sequence. By tuning the motion profile to maximize the retract phase relative to the advance phase, the capsule creates a controlled friction anisotropy at the interface that generates net forward thrust. We developed an analytical model incorporating a Kelvin-Voigt formulation to capture the viscoelastic stick–slip interactions between the sliders and the tissue, linking the asymmetry between advance and retract phase durations to mean thrust, and slider-reversal synchronization to thrust stability. Comprehensive force characterization experiments in ex-vivo porcine colon revealed a mean steady-state traction force of 0.85 N, closely matching the model. Furthermore, experiments confirmed that thrust generation is speed-independent and scales linearly with the phase asymmetry, in agreement with theoretical predictions, underscoring the capsule's predictable performance and scalability. In locomotion validation experiments, OSCAR demonstrated robust performance, achieving an average speed of 3.08 mm/s, a velocity sufficient to match the cecal intubation times of conventional colonoscopy. By coupling phase-encoded friction anisotropy with a predictive model, OSCAR delivers controllable thrust generation at low normal loads, enabling safer and more robust self-propelling locomotion for robotic capsule colonoscopy.
Read the full abstract ▾
All authors Mostafa A. Atalla, Anand S. Sekar, Remi van Starkenburg, David J. Jager, Aimée Sakes, Michaël Wiertlewski, Paul Breedveld