We present Splat-Nav, a real-time robot navigation pipeline designed to work with environment representations generated by Gaussian Splatting (GSplat), a powerful new 3D scene representation. Splat-Nav consists of two components: 1) Splat-Plan, a safe planning module, and 2) Splat-Loc, a robust vision-based pose estimation module. Splat-Plan builds a safe-by-construction polytope corridor through the map based on mathematically rigorous collision constraints and then constructs a Bézier curve trajectory through this corridor. Splat-Loc provides real-time recursive state estimates given only an RGB feed from an on-board camera, leveraging the point-cloud representation inherent in GSplat scenes. Splat-Nav endows robots the ability to recursively re-plan smooth and safe trajectories to goal locations. Goal locations can be specified with position coordinates, or with language commands by using a language embedded GSplat. We demonstrate the safety and robustness of our pipeline in both simulation and hardware experiments, where we show online re-planning at more than 2 Hz and pose estimation at about 25 Hz, an order of magnitude faster than Neural Radiance Field (NeRF)-based navigation methods, thereby enabling real-time navigation.
Four objects within the scene (beachball, keyboard, microwave, phonebook) are semnatically segmented and chosen as goal locations for our trials. We run using three control schemes (Open-Loop, Closed-Loop VIO, and Splat-Loc) each demonstrated in the videos below. We refer readers to our paper.
Open-loop schemes do not re-plan, relying on the onboard VIO to track the trajectory.
Closed-loop uses VIO to estimate poses and re-plan accordingly.
Splat-Loc estimates the current drone pose and re-plans a path to the goal location. This method is robust against disturbance and drift.
We also demonstrate the robustness of Splat-Plan by tracking open-loop trajectories at 1.5 m/s.
Finally, closed-loop re-planning using Splat-Plan and Splat-Loc can safely navigate cluttered environments over long periods of time.
@misc{chen2024splatnav,
title={Splat-Nav: Safe Real-Time Robot Navigation in Gaussian Splatting Maps},
author={Timothy Chen and Ola Shorinwa and Joseph Bruno and Aiden Swann and Javier Yu and Weijia Zeng and Keiko Nagami and Philip Dames and Mac Schwager},
year={2024},
eprint={2403.02751},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2403.02751},
}