Robotics
Safety Guardrails in the Sky: Realizing Control Barrier Functions on the VISTA F-16 Jet
Andrew W. Singletary, Max H. Cohen, Tamas G. Molnar et al. (+1 more)
The advancement of autonomous systems – from legged
robots to self-driving vehicles and aircraft – necessitates executing increasingly high-performance and dynamic motions without ever putting the system or its environment in harm's way. In this paper, we introduce Guardrails – a novel runtime assurance mechanism that guarantees dynamic safety for autonomous systems, allowing them to safely evolve on the edge of their operational domains. Rooted in the theory of control barrier functions, Guardrails offers a control strategy that carefully blends commands from a human or AI operator with safe control actions to guarantee safe behavior. To demonstrate its capabilities, we implemented Guardrails on an F-16 fighter jet and conducted flight tests where Guardrails supervised a human pilot to enforce g-limits, altitude bounds, geofence constraints, and combinations thereof. Throughout extensive flight testing, Guardrails successfully ensured safety, keeping the pilot in control when safe to do so and minimally modifying unsafe pilot inputs otherwise.
Read the full abstract ▾
All authors Andrew W. Singletary, Max H. Cohen, Tamas G. Molnar, Aaron D. Ames
Robotics · Cross-listed from eess.SY
An Empirical Markov Chain Car-Following (MC-CF) Model
Sungyong Chung, Yanlin Zhang, Nachuan Li et al. (+2 more)
Car-following behavior is fundamental to traffic flow theory, yet traditional models often fail to capture the stochasticity of naturalistic driving. This paper proposes an empirical probabilistic sampling approach to car-following modeling that bypasses conventional parametric assumptions. Under this approach, we introduce the Markov Chain Car-Following (MC-CF) model, which represents state transitions as a Markov process and predicts behavior by randomly sampling accelerations from empirical distributions within discretized state bins. Evaluation on the Waymo Open Motion
Dataset (WOMD) demonstrates that MC-CF variants significantly outperform all physics-based baselines (IDM, Gipps, FVDM, and SIDM) across both one-step and open-loop trajectory prediction metrics, and remain competitive with modern data-driven baselines including neural network and Gaussian mixture model approaches. Zero-shot generalization on the Naturalistic Phoenix (PHX)
dataset further confirms cross-domain transferability. Finally, microscopic ring road simulations validate the framework's scalability: by incrementally integrating unconstrained free-flow trajectories and high-speed freeway data (TGSIM) alongside a conservative inference strategy, the model substantially reduces collisions across most tested scenarios and successfully reproduces naturalistic and
stochastic shockwave propagation, though crashes persist under severe shockwave conditions. Overall, the proposed MC-CF model provides a robust and scalable foundation for simulating population-level
stochastic traffic behavior that requires no behavioral parameter calibration, making it well-suited for the data-rich future of intelligent transportation.
Read the full abstract ▾
All authors Sungyong Chung, Yanlin Zhang, Nachuan Li, Dana Monzer, Alireza Talebpour
Robotics · Cross-listed from eess.SY
MPC as a Copilot: A Predictive Filter Framework with Safety and Stability Guarantees
Yunda Yan, Chenxi Tao, Jinya Su et al. (+2 more)
Ensuring both safety and stability remains a fundamental challenge in learning-based control, where goal-oriented policies often neglect system constraints and closed-loop state convergence. To address this limitation, this paper introduces the Predictive Safety–Stability Filter (PS2F), a unified predictive filter framework that guarantees constraint satisfaction and asymptotic stability within a single architecture. The PS2F framework comprises two cascaded optimal control problems: a nominal model predictive control (MPC) layer that serves solely as a copilot, implicitly defining a Lyapunov function and generating safety- and stability-certified predicted trajectories, and a secondary filtering layer that adjusts external command to remain within a provably safe and stable region. This cascaded structure enables PS2F to inherit the theoretical guarantees of nominal MPC while accommodating goal-oriented external commands. Rigorous analysis establishes recursive feasibility and asymptotic stability of the closed-loop system without introducing additional conservatism beyond that associated with the nominal MPC. Furthermore, a time-varying parameterisation allows PS2F to transition smoothly between safety-prioritised and stability-oriented operation modes, providing a principled mechanism for balancing exploration and exploitation. The effectiveness of the proposed framework is demonstrated through comparative numerical experiments.
Read the full abstract ▾
All authors Yunda Yan, Chenxi Tao, Jinya Su, Cunjia Liu, Shihua Li
Robotics · Cross-listed from cs.LG
Kernel Dynamics under Path Entropy Maximization
Jnaneshwar Das
We propose a variational framework in which the kernel function k : X x X -> R, interpreted as the foundational object encoding what distinctions an
agent can represent, is treated as a dynamical variable subject to path
entropy maximization (Maximum Caliber, MaxCal). Each kernel defines a representational structure over which an information geometry on probability space may be analyzed; a trajectory through kernel space therefore corresponds to a trajectory through a family of effective geometries, making the optimization landscape endogenous to its own traversal. We formulate fixed-point conditions for self-consistent kernels, propose renormalization group (RG) flow as a structured special case, and suggest neural tangent kernel (NTK)
evolution during deep network training as a candidate empirical instantiation. Under explicit information-thermodynamic assumptions, the work required for kernel change is bounded below by delta W >= k_B T delta I_k, where delta I_k is the mutual information newly unlocked by the updated kernel. In this view, stable fixed points of MaxCal over kernels correspond to self-reinforcing distinction structures, with biological niches, scientific paradigms, and craft mastery offered as conjectural interpretations. We situate the framework relative to assembly theory and the MaxCal literature, separate formal results from structured correspondences and conjectural bridges, and pose six open questions that make the program empirically and mathematically testable.
Read the full abstract ▾
Robotics · Cross-listed from math.OC
Separation is Optimal for LQR under Intermittent Feedback
Abdullah Y. Etcibasi, C. Emre Koksal and Eylem Ekici
We study finite-horizon linear-quadratic regulation of a scalar linear system with intermittent state feedback under an average communication-rate constraint. In this setting, the scheduling policy and controller are generally coupled through the dual effect: transmission decisions shape future estimation errors, while control actions influence the information available for scheduling. Existing treatments often recover tractability by restricting
attention to symmetric scheduling policies, but the optimality of this restriction has remained unclear. We show that, for i.i.d. zero-mean disturbances, symmetric policies are optimal. Consequently, the communication-constrained LQR problem admits a separation structure. The optimal controller is a linear feedback law independent of the scheduling policy, while the optimal scheduler is obtained from a dynamic program. We further show that the optimal scheduling rule is a symmetric threshold policy in the accumulated disturbance since the most recent update.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Benchmarking Multi-View BEV Object Detection with Mixed Pinhole and Fisheye Cameras
Xiangzhong Liu and Hao Shen
Modern autonomous driving systems increasingly rely on mixed camera configurations with pinhole and fisheye cameras for full view perception. However, Bird's-Eye View (BEV) 3D object detection models are predominantly designed for pinhole cameras, leading to performance degradation under fisheye distortion. To bridge this gap, we introduce a multi-view BEV detection
benchmark with mixed cameras by converting KITTI-360 into nuScenes format. Our study encompasses three adaptations: rectification for zero-shot evaluation and fine-tuning of nuScenes-trained models, distortion-aware view transformation modules (VTMs) via the MEI camera model, and polar coordinate representations to better align with radial distortion. We systematically evaluate three representative BEV architectures, BEVFormer, BEVDet and PETR, across these strategies. We demonstrate that projection-free architectures are inherently more robust and effective against fisheye distortion than other VTMs. This work establishes the first real-data 3D detection
benchmark with fisheye and pinhole images and provides systematic adaptation and practical guidelines for designing robust and cost-effective 3D perception systems. The code is available at https://github.com/CesarLiu/FishBEVOD.git.
Read the full abstract ▾
Robotics
Probe-to-Grasp Manipulation Using Self-Sensing Pneumatic Variable-Stiffness Joints
Ngoc Duy Tran, Yeman Fan, Feng Dai et al. (+5 more)
Grasping deformable objects with varying stiffness remains a significant challenge in robotics. Estimating the local stiffness of a target object is important for determining an optimal grasp pose that enables stable pickup without damaging the object. This paper presents a probe-to-grasp manipulation framework for estimating the relative stiffness of objects using a passive soft-rigid two-finger hybrid gripper equipped with self-sensing pneumatic variable-stiffness joints. Each finger of the gripper consists of two rigid links connected by a soft pneumatic ring placed at the joint, enabling both compliant interaction and controllable joint stiffness via internal pressurization. By measuring the pressure inside the pneumatic ring, we can estimate the interaction force during contact. Building on this, we propose a practical probing strategy to infer relative object stiffness by correlating the estimated normal force with known gripper closing displacement. We validate the self-sensing model through stiffness characterization experiments across bending angles and pressure ranges, and demonstrate stiffness-aware probing-and-grasping in real-life applications: selecting grasp locations on fruits with spatially varying stiffness. The proposed system offers a minimal, low-cost sensing approach for stiffness-aware soft manipulation while retaining probing and grasping capability.
Read the full abstract ▾
All authors Ngoc Duy Tran, Yeman Fan, Feng Dai, Khang Nguyen, Anh Nguyen, Hoang Hiep Ly, Tung D. Ta, Shigeru Chiba
Robotics · Cross-listed from cs.GR
Engineering Mythology: A Digital-Physical Framework for Culturally-Inspired Public Art
Jnaneshwar Das, Christopher Filkins, Rajesh Moharana et al. (+13 more)
Navagunjara Reborn: The Phoenix of Odisha was built for Burning Man 2025 as both a sculpture and an experiment-a fusion of myth, craft, and computation. This paper describes the digital-physical workflow developed for the project: a pipeline that linked digital sculpting, distributed fabrication by artisans in Odisha (India), modular structural optimization in the U.S., iterative feedback through photogrammetry and digital twins, and finally, one-shot full assembly at the art site in Black Rock Desert, Nevada. The desert installation tested not just materials, but also systems of collaboration: between artisans and engineers, between myth and technology, between cultural specificity and global experimentation. We share the lessons learned in design, fabrication, and deployment and offer a framework for future interdisciplinary projects at the intersection of cultural heritage, STEAM education, and public art. In retrospect, this workflow can be read as a convergence of many knowledge systems-artisan practice, structural engineering, mythic narrative, and environmental constraint-rather than as execution of a single fixed blueprint.
Read the full abstract ▾
All authors Jnaneshwar Das, Christopher Filkins, Rajesh Moharana, Ekadashi Barik, Bishweshwar Das, David Ayers, Christopher Skiba, Rodney Staggers Jr, Mark Dill, Swig Miller, Daniel Tulberg, Patrick Smith, Seth Brink, Kyle Breen, Harish Anand, Ramon Arrowsmith
Robotics
Which Reconstruction Model Should a Robot Use? Routing Image-to-3D Models for Cost-Aware Robotic Manipulation
Akash Anand, Aditya Agarwal and Leslie Pack Kaelbling
Robotic manipulation tasks require 3D mesh reconstructions of varying quality: dexterous manipulation demands fine-grained surface detail, while collision-free planning tolerates coarser representations. Multiple reconstruction methods offer different cost-quality tradeoffs, from Image-to-3D models - whose output quality depends heavily on the input viewpoint - to view-invariant methods such as structured light scanning. Querying all models is computationally prohibitive, motivating per-input model selection. We propose SCOUT, a novel routing framework that decouples reconstruction scores into two components: (1) the relative performance of viewpoint-dependent models, captured by a learned probability distribution, and (2) the overall image difficulty, captured by a scalar partition function estimate. As the learned network operates only over the viewpoint-dependent models, view-invariant pipelines can be added, removed, or reconfigured without retraining. SCOUT also supports arbitrary cost constraints at inference time, accommodating the multi-dimensional cost constraints common in robotics. We evaluate on the Google Scanned Objects, BigBIRD, and YCB
datasets under multiple mesh quality metrics, demonstrating consistent improvements over routing baselines adapted from the LLM literature across various cost constraints. We further validate the framework through robotic grasping and dexterous manipulation experiments. We release the code and additional results on our website.
Read the full abstract ▾
Robotics
Spectral Decomposition of Inverse Dynamics for Fast Exploration in Model-Based Manipulation
Solvin Sigurdson, Benjamin Riviere and Joel Burdick
Planning long duration robotic manipulation sequences is challenging because of the complexity of exploring feasible trajectories through nonlinear contact dynamics and many contact modes. Moreover, this complexity grows with the problem's horizon length. We propose a search tree method that generates trajectories using the spectral decomposition of the inverse dynamics equation. This equation maps actuator displacement to object displacement, and its spectrum is efficient for exploration because its components are orthogonal and they approximate the reachable set of the object while remaining dynamically feasible. These trajectories can be combined with any search based method, such as Rapidly-Exploring Random Trees (RRT), for long-horizon planning. Our method performs similarly to recent work in model-based planning for short-horizon tasks, and differentiates itself with its ability to solve long-horizon tasks: whereas existing methods fail, ours can generate 45 second duration, 10+ contact mode plans using 15 seconds of computation, demonstrating real-time capability in highly complex domains.
Read the full abstract ▾
Robotics
Transferability Through Cooperative Competitions
Rodrigo Serra, Carlos Azevedo, André Silva et al. (+6 more)
This paper presents a novel framework for cooperative robotics competitions (coopetitions) that promote the transferability and composability of robotics modules, including software, hardware, and data, across heterogeneous robotic systems. The framework is designed to incentivize collaboration between teams through structured task design, shared infrastructure, and a royalty-based scoring system. As a case study, the paper details the implementation and outcomes of the first euROBIN Coopetition, held under the European Robotics and AI Network (euROBIN), which featured fifteen robotic platforms competing across Industrial, Service, and Outdoor domains. The study highlights the practical challenges of achieving module reuse in real-world scenarios, particularly in terms of integration complexity and system compatibility. It also examines participant performance, integration behavior, and team feedback to assess the effectiveness of the framework. The paper concludes with lessons learned and recommendations for future coopetitions, including improveme
Read the full abstract ▾
All authors Rodrigo Serra, Carlos Azevedo, André Silva, Kevin Alcedo, Quentin Rouxel, Peter So, Alejandro Suarez, Alin Albu-Schäeffer, Pedro U. Lima
Robotics · Cross-listed from cs.CV
E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences
Biswadeep Sen, Benoit R. Cottereau, Nicolas Cuperlier et al. (+1 more)
Event-based cameras capture visual information as asynchronous streams of per-pixel brightness changes, generating sparse, temporally precise data. Compared to conventional frame-based sensors, they offer significant advantages in capturing high-speed dynamics while consuming substantially less power. Predicting future event representations from past observations is an important problem, enabling downstream tasks such as future semantic
segmentation or object tracking without requiring access to future sensor measurements. While recent state-of-the-art approaches achieve strong performance, they often rely on computationally heavy backbones and, in some cases, large-scale pretraining, limiting their applicability in resource-constrained scenarios. In this work, we introduce E-TIDE, a lightweight, end-to-end trainable architecture for event-tensor prediction that is designed to operate efficiently without large-scale pretraining. Our approach employs the TIDE module (Temporal Interaction for Dynamic Events), motivated by efficient spatiotemporal interaction design for sparse event tensors, to capture temporal dependencies via large-kernel mixing and activity-aware gating while maintaining low computational complexity. Experiments on standard event-based
datasets demonstrate that our method achieves competitive performance with significantly reduced model size and training requirements, making it well-suited for real-time deployment under tight latency and
memory budgets.
Read the full abstract ▾
All authors Biswadeep Sen, Benoit R. Cottereau, Nicolas Cuperlier, Terence Sim
Robotics
Heracles: Bridging Precise Tracking and Generative Synthesis for General Humanoid Control
Zelin Tao, Zeran Su, Peiran Liu et al. (+13 more)
Achieving general-purpose
humanoid control requires a delicate balance between the precise execution of commanded motions and the flexible, anthropomorphic adaptability needed to recover from unpredictable environmental perturbations. Current general controllers predominantly formulate motion control as a rigid reference-tracking problem. While effective in nominal conditions, these trackers often exhibit brittle, non-anthropomorphic failure modes under severe disturbances, lacking the generative adaptability inherent to human motor control. To overcome this limitation, we propose Heracles, a novel state-conditioned
diffusion middleware that bridges precise motion tracking and generative synthesis. Rather than relying on rigid tracking paradigms or complex explicit mode-switching, Heracles operates as an intermediary layer between high-level reference motions and low-level physics trackers. By conditioning on the
robot's real-time state, the
diffusion model implicitly adapts its behavior: it approximates an identity map when the state closely aligns with the reference, preserving zero-shot tracking fidelity. Conversely, when encountering significant state deviations, it seamlessly transitions into a generative synthesizer to produce natural, anthropomorphic recovery trajectories. Our framework demonstrates that integrating generative priors into the control loop not only significantly enhances robustness against extreme perturbations but also elevates
humanoid control from a rigid tracking paradigm to an open-ended, generative general-purpose architecture.
Read the full abstract ▾
All authors Zelin Tao, Zeran Su, Peiran Liu, Jingkai Sun, Wenqiang Que, Jiahao Ma, Jialin Yu, Jiahang Cao, Pihai Sun, Hao Liang, Gang Han, Wen Zhao, Zhiyuan Xu, Jian Tang, Qiang Zhang, Yijie Guo
Robotics
TerraSkipper: A Centimeter-Scale Robot for Multi-Terrain Skipping and Crawling
Shashwat Singh, Sheri Zhang, Spencer Matonis et al. (+1 more)
Mudskippers are unique amphibious fish capable of locomotion in diverse environments, including terrestrial surfaces, aquatic habitats, and highly viscous substrates such as mud. This versatile locomotion is largely enabled by their powerful tail, which stores and rapidly releases energy to produce impulsive jumps. Inspired by this biological mechanism, we present the design and development of a multi-terrain centimeter-scale skipping and crawling
robot. The
robot is predominantly 3D printed and features onboard sensing, computation, and power. It is equipped with two side fins for crawling, each integrated with a
hall effect sensor for gait control, while a rotary springtail driven by a 10mm planetary gear motor enables continuous impulsive skipping across a range of substrates to achieve multi-terrain locomotion. We modeled and experimentally characterized the tail, identifying an optimal length of 25mm that maximizes the mean propulsive force (4N, peaks up to 6N) for forward motion. In addition, we evaluated skipping on substrates where fin based crawling alone fails, and varied the moisture content of uniform sand and bentonite clay powder to compare skipping with crawling. Skipping consistently produced higher mean velocities than crawling, particularly on viscous and granular media. Finally, outdoor tests on grass, loose sand, and hard ground confirmed that combining skipping on entangling and granular terrain with crawling on firm ground extends the operational range of the
robot in real-world environments.
Read the full abstract ▾
All authors Shashwat Singh, Sheri Zhang, Spencer Matonis, Zeynep Temel
Robotics
ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation
Hongyu Yan, Qiwei Li, Jiaolong Yang et al. (+1 more)
Most existing
vision-
language-action (VLA) models for robotic manipulation lack progress awareness, typically relying on hand-crafted heuristics for task termination. This limitation is particularly severe in long-horizon tasks involving cascaded sub-goals. In this work, we investigate the estimation and integration of task progress, proposing a novel model named {\textbf \vla}. Our technical contributions are twofold: (1) robust progress estimation: We pre-train a progress
estimator on large-scale, unsupervised
video-text robotic
datasets. This
estimator achieves a low prediction residual (0.07 on a scale of $[0, 1]$) in simulation and demonstrates zero-shot generalization to unseen real-world samples, and (2) differentiable progress guidance: We introduce an inverse dynamics world model that maps predicted action
tokens into future latent visual states. These latents are then processed by the progress
estimator; by applying a maximal progress regularization, we establish a differentiable pipeline that provides progress-piloted guidance to refine action
tokens. Extensive experiments on the CALVIN and LIBERO
benchmarks, alongside real-world
robot deployment, consistently demonstrate substantial improvements in success rates and generalization over strong baselines.
Read the full abstract ▾
All authors Hongyu Yan, Qiwei Li, Jiaolong Yang, Yadong Mu
Robotics
ContraMap: Contrastive Uncertainty Mapping for Robot Environment Representation
Chi Cuong Le and Weiming Zhi
Reliable
robot perception requires not only predicting scene structure, but also identifying where predictions should be treated as unreliable due to sparse or missing observations. We present ContraMap, a contrastive continuous mapping method that augments kernel-based discriminative maps with an explicit uncertainty class trained using synthetic noise samples. This formulation treats unobserved regions as a contrastive class, enabling joint environment prediction and spatial uncertainty estimation in real time without
Bayesian inference. Under a simple mixture-model view, we show that the probability assigned to the uncertainty class is a monotonic function of a distance-aware uncertainty surrogate. Experiments in 2D occupancy mapping, 3D semantic mapping, and tabletop scene reconstruction show that ContraMap preserves mapping quality, produces spatially coherent uncertainty estimates, and is substantially more efficient than
Bayesian kernelmap baselines.
Read the full abstract ▾
Robotics
LLM-Enabled Low-Altitude UAV Natural Language Navigation via Signal Temporal Logic Specification Translation and Repair
Yuqi Ping, Huahao Ding, Tianhao Liang et al. (+6 more)
Natural
language (NL)
navigation for low-altitude unmanned aerial vehicles (
UAVs) offers an intelligent and convenient solution for low-altitude aerial services by enabling an intuitive interface for non-expert operators. However, deploying this capability in urban environments necessitates the precise grounding of underspecified instructions into safety-critical, dynamically feasible motion plans subject to spatiotemporal constraints. To address this challenge, we propose a unified framework that translates NL instructions into Signal Temporal Logic (STL) specifications and subsequently synthesizes trajectories via mixed-integer linear programming (MILP). Specifically, to generate executable STL formulas from free-form NL, we develop a
reasoning-enhanced large
language model (LLM) leveraging chain-of-thought (CoT) supervision and group-relative policy optimization (GRPO), which ensures high syntactic validity and semantic consistency. Furthermore, to resolve infeasibilities induced by stringent logical or spatial requirements, we introduce a specification repair mechanism. This module combines MILP-based diagnosis with LLM-guided semantic
reasoning to selectively relax task constraints while strictly enforcing safety guarantees. Extensive simulations and real-world flight experiments demonstrate that the proposed closed-loop framework significantly improves NL-to-STL translation robustness, enabling safe, interpretable, and adaptable
UAV navigation in complex scenarios.
Read the full abstract ▾
All authors Yuqi Ping, Huahao Ding, Tianhao Liang, Longyu Zhou, Guangyu Lei, Xinglin Chen, Junwei Wu, Jieyu Zhou, Tingting Zhang
Robotics · Cross-listed from cs.CV
Daojie Peng, Fulong Ma and Jun Ma
Vision-
Language Navigation (VLN) requires an
embodied agent to navigate complex environments by following natural
language instructions, which typically demands tight fusion of visual and
language modalities. Existing VLN methods often convert raw images into visual
tokens or implicit features, requiring large-scale visual pre-training and suffering from poor generalization under environmental variations (e.g., lighting, texture). To address these issues, we propose SOL-Nav (Structured Observation
Language for
Navigation), a novel framework that translates
egocentric visual observations into compact structured
language descriptions for efficient and generalizable
navigation. Specifically, we divide RGB-D images into a NxN grid, extract representative semantic, color, and depth information for each grid
cell to form structured text, and concatenate this with the
language instruction as pure
language input to a pre-trained
language model (PLM). Experimental results on standard VLN
benchmarks (R2R, RxR) and real-world deployments demonstrate that SOL-Nav significantly reduces the model size and training data dependency, fully leverages the
reasoning and representation capabilities of PLMs, and achieves strong generalization to unseen environments.
Read the full abstract ▾
Robotics
Learning Smooth and Robust Space Robotic Manipulation of Dynamic Target via Inter-frame Correlation
Siyi Lang, Hongyi Gao, Yingxin Zhang et al. (+5 more)
On-orbit servicing represents a critical frontier in future aerospace engineering, with the manipulation of dynamic non-cooperative targets serving as a key technology. In microgravity environments, objects are typically free-floating, lacking the support and frictional constraints found on Earth, which significantly escalates the complexity of tasks involving space robotic manipulation. Conventional planning and control-based methods are primarily limited to known, static scenarios and lack real-time responsiveness. To achieve precise robotic manipulation of dynamic targets in unknown and unstructured space environments, this letter proposes a data-driven space robotic manipulation approach that integrates historical temporal information and inter-frame correlation mechanisms. By exploiting the temporal correlation between historical and current frames, the system can effectively capture motion features within the scene, thereby producing stable and smooth manipulation trajectories for dynamic targets. To validate the effectiveness of the proposed method, we developed a ground-based experimental platform consisting of a PIPER X robotic arm and a dual-axis linear stage, which accurately simulates micro-
gravity free-floating motion in a 2D plane.
Read the full abstract ▾
All authors Siyi Lang, Hongyi Gao, Yingxin Zhang, Zihao Liu, Hanlin Dong, Zhaoke Ning, Zhiqiang Ma, Panfeng Huang
Robotics
S3KF: Spherical State-Space Kalman Filtering for Panoramic 3D Multi-Object Tracking
Zhongyuan Liu, Shaonan Yu, Jianping Li et al. (+5 more)
Panoramic multi-object tracking is important for industrial safety monitoring, wide-area robotic perception, and infrastructure-light deployment in large workspaces. In these settings, the sensing system must provide full-surround coverage, metric
geometric cues, and stable target association under wide field-of-view distortion and occlusion. Existing image-plane trackers are tightly coupled to the camera projection and become unreliable in panoramic imagery, while conventional Euclidean 3D formulations introduce redundant directional parameters and do not naturally unify angular, scale, and depth estimation. In this paper, we present $\mathbf{S^3KF}$, a panoramic 3D multi-object tracking framework built on a motorized rotating
LiDAR and a quad-fisheye camera rig. The key idea is a geometry-consistent state representation on the unit sphere $\mathbb{S}^2$, where object bearing is modeled by a two-degree-of-freedom tangent-plane parameterization and jointly estimated with box scale and depth dynamics. Based on this state, we derive an extended spherical Kalman filtering pipeline that fuses panoramic camera detections with
LiDAR depth observations for
multimodal tracking. We further establish a map-based ground-truth generation pipeline using wearable localization devices registered to a shared global
LiDAR map, enabling quantitative evaluation without motion-capture infrastructure. Experiments on self-collected real-world sequences show decimeter-level planar tracking accuracy, improved identity continuity over a 2D panoramic baseline in dynamic scenes, and real-time onboard operation on a Jetson AGX Orin platform. These results indicate that the proposed framework is a practical solution for panoramic perception and industrial-scale multi-object tracking.The project page can be found at https://kafeiyin00.github.io/S3KF/.
Read the full abstract ▾
All authors Zhongyuan Liu, Shaonan Yu, Jianping Li, Pengfei Wan, Xinhang Xu, Pengfei Wang, Maggie Y. Gao, Lihua Xie
Robotics
Copilot-Assisted Second-Thought Framework for Brain-to-Robot Hand Motion Decoding
Yizhe Li, Shixiao Wang and Jian K. Liu
Motor kinematics prediction (MKP) from electroencephalography (
EEG) is an important research area for developing movement-related
brain-computer interfaces (BCIs). While traditional methods often rely on convolutional neural networks (CNNs) or recurrent neural networks (RNNs),
Transformer-based models have shown strong ability in modeling long sequential
EEG data. In this study, we propose a CNN-
attention hybrid model for decoding hand kinematics from
EEG during grasp-and-lift tasks, achieving strong performance in within-subject experiments. We further extend this approach to
EEG-EMG
multimodal decoding, which yields substantially improved results. Within-subject tests achieve PCC values of 0.9854, 0.9946, and 0.9065 for the X, Y, and Z axes, respectively, computed on the midpoint trajectory between the thumb and index finger, while cross-subject tests result in 0.9643, 0.9795, and 0.5852. The decoded trajectories from both modalities are then used to control a Franka Panda robotic arm in a MuJoCo simulation. To enhance trajectory fidelity, we introduce a copilot framework that filters low-confidence decoded points using a motion-state-aware critic within a finite-state machine. This post-processing step improves the overall within-subject PCC of
EEG-only decoding to 0.93 while excluding fewer than 20% of the data points.
Read the full abstract ▾
Robotics
Robotic Dexterous Manipulation via Anisotropic Friction Modulation using Passive Rollers
Ethan Fisk, Taeyoon Lee and Shenli Yuan
Controlling friction at the fingertip is fundamental to dexterous manipulation, yet remains difficult to realize in robotic hands. We present the design and analysis of a robotic fingertip equipped with passive rollers that can be selectively braked or pivoted to modulate contact friction and constraint directions. When unbraked, the rollers permit unconstrained sliding of the contact point along the rolling direction; when braked, they resist motion like a conventional fingertip. The rollers are mounted on a pivoting mechanism, allowing reorientation of the constraint frame to accommodate different manipulation tasks. We develop a constraint-based model of the fingertip integrated into a parallel-jaw gripper and analyze its ability to support diverse manipulation strategies. Experiments show that the proposed design enables a wide range of dexterous actions that are conventionally challenging for robotic grippers, including sliding and pivoting within the grasp, robust adaptation to uncertain contacts, multi-object or multi-part manipulation, and interactions requiring asymmetric friction across fingers. These results demonstrate the versatility of passive roller fingertips as a low-complexity, mechanically efficient approach to friction modulation, advancing the development of more adaptable and robust robotic manipulation.
Read the full abstract ▾