FoundLoc: GNSS-Denied UAV Localization

FoundLoc estimates a UAV’s position when GNSS is unavailable by combining a downward-facing camera, an IMU, and preloaded satellite imagery. Visual place recognition supplies map matches, while VIO estimates relative motion; map alignment and an extended Kalman filter combine these into a global position estimate.

I worked on the project as a full-time Robotics Researcher at CMU’s AirLab, under Dr. Sebastian Scherer, in collaboration with the ViSafe detect-and-avoid effort. A central achievement was running the localization pipeline, including its pretrained vision model, onboard a custom sensor payload during real flights.

Localization software and model deployment

I led the localization architecture and implementation, with major involvement in reference-image preparation, visual place recognition, map alignment, filtering, and evaluation. I wrote the C++/Python onboard pipeline and ROS drivers, integrating the team’s VIO component; the underlying VIO algorithm was developed by collaborators.

We initially trained a GAN-based VPR approach similar to my master’s research, then moved to off-the-shelf DINO features through AnyLoc because they performed better without task-specific training or fine-tuning. I adapted the model and pipeline to run inside a Docker container on an NVIDIA Jetson Xavier NX, working within the module’s limited compute resources.

Payload integration and flight testing

The sensor unit was adapted from an earlier lab project. I helped select its optics and designed mounting and power arrangements for both the original DJI Matrice testbed and the final NDAA-compliant hexacopter. I connected payload engineering with onboard software development, drone data collection, and flight testing. Across the shared localization and detect-and-avoid campaign, I accumulated 80+ hours as a safety pilot, including data collection, field tests, and the final demonstration.

The FoundLoc paper reports 19.38 m average absolute trajectory error across its combined Nardo-Air evaluation. On the Jetson Xavier NX, the team reported 8 Hz VIO and approximately 1.93 Hz DINO-based VPR inference. These are separate component rates, demonstrating onboard execution within the payload’s compute limits.

The evaluation covered changes in season, lighting, weather, altitude, and flight pattern. Tests used north-aligned imagery and fixed-altitude flight segments; altitude-variation experiments had higher error, partly attributed to uncorrected camera tilt. The measurements describe the complete team’s system.