Close
Do you have any questions? Contact us!
Blog
Blog

Visual Positioning System (VPS): What’s under the hood

One of the skills of human pilots we are replicating to achieve the real autonomous flight is the ability to navigate based purely on vision, without relying on GPS or even stereo cameras.

GPS (or, more broadly, GNSS) is the benchmark for navigation aid, and the world expects a comparable experience from its alternative. But the visual-based positioning requires effort to tune it up to the operational strength. This fact surprises some of our clients, so we would like to address it today.

How it works


Daedalean’s Visual Positioning System relies on a map to localise itself within a global coordinate frame. This map comprises descriptors of visual features. While recognising these features in the imagery from the onboard camera during flight, the system defines their positions in three dimensions using the WGS84 coordinate frame. From that, it computes an accurate estimate of its own position and attitude.

Daedalean’s explanative visualisation of features over terrain: this is not what a pilot sees on the system display, but reconstructed from the system’s “map” files


You can’t localise yourself on a map without, well… the map! Before the first use of the VPS, it needs to construct a map from the imagery surveyed in the presence of a good GNSS signal. Later, an amount of features recognised in subsequent flights provides a good measure of the map quality and the expected precision of the localisation. It is also necessary to monitor the map for degradation over time.

The map does not necessarily contain features obvious to the human eye. Instead, these are features that the computer can reliably distinguish from an often noisy background. 

Where do the maps come from?


From the customer’s perspective, a map for necessary terrain comes with the system. 

From Daedalean’s angle, the process of creating a map is actually a pre-survey. The VPS software contains a specific component for processing maps during survey flights (done by a customer or us). This component processes the imagery obtained from the VPS camera, detecting features to be used as landmarks, and binds coordinates to them. The result is a data map, recognisable by the navigation component of the system but not intended for a cockpit display. Using this map, the system then localises the aircraft during regular flights without the need for GNSS or other means of navigation. 

Pre-surveying flights are costly, and here is where simulation comes to the rescue. We have a dedicated in-house team working with various sources of terrain and other geographical data. These sources are, in particular: a) simulator engines rendering landscapes (such as Unigine) from our GIS framework, and b) providers of real orthophoto – aerial photo imagery with geometrical proportions corrected to fit a map projection (e.g. USGS and Swisstopo, and here we’d like to note that we highly appreciate working with the latter). So, our engineers are working on covering the areas of interest with more and more detailed synthetic maps. And thanks to this, we are ramping up the capability to create maps out of an extensive database of reality, sourced in a highly scalable way for the whole planet. 

Screenshot made on UNIGINE



Aerial photograph of Kandersteg (centred on the main church's bell tower). Made by the Federal Office of Topography (swisstopo). Source: Wikimedia


Note: It may be possible in the future to extend the system to the point where you could feed it an existing large-scale aeronautical map, and it could recognise the relatively few but highly abstracted features represented on it for the purpose of flying at higher altitudes.

The problem of maps longevity and its solution


During military exercises, a dogface can’t find the point where he’d spotted the “enemy” dugout. “You were instructed to determine two landmarks next to the object!” - “I did! But then the landmark number two has ridden the landmark number one, and they left, Sir!”
An old army joke

Mount Matterhorn in Switzerland looked quite similar 10 million years ago. The Eiffel Tower hasn’t changed much since 100 years ago. Manhattan will have most of the landmarks it had 20-50 years ago. But, for example, Dubai may be trickier. Its skyline keeps changing, and you will not recognise some familiar parts of the city after a year of absence.

And last February, after the heavy snow in Switzerland, we discovered how different the roundabouts of Zurich could suddenly look! There are terrains with substantial seasonal changes, and we should ensure that a “visual” map valid a week ago is still good a week after.

And even this is not the limit: there are daily changes in lighting that influence the recognition of terrain.

Barcelona night and day. Photo by @willcheyney on Instagram, source: https://www.instagram.com/p/Bubs9z4HNrc


Our features are optimised for longevity – i.e. engineered to maximise the probability that a feature is recognisable in the same place a long time after it was first recorded. We have only started collecting the relevant imagery at the beginning of 2020, so we do not yet have sufficient data to guarantee, let alone certify, their life expectancy. However, simulations give us high expectations; and maps from March 2020 continued to work throughout the remainder of the same year. After working on reversing the changes we observed since that, we are sure that we can make a composite all-year map based on features that work well throughout at least part of the year and with a sufficient number of features recognisable at any given time.

By the way, the GPS almanack is valid with little dilution of precision for up to two weeks but is considered valid for up to 180 days.

Maintenance to keep maps up-to-date


After a customer installs the VPS with a map inside, what happens next?

During their regular flights, they anyway collect data on the changes of the area of flying: either turning on the survey mode of the system in flight or merely recording the same imagery for the later offline processing. This data is sufficient to update the system maps, but the process is not that simple. 

It is a DO-200-certified process, which means it involves rigorous quality assurance before the updated map is rolled out to the fleet. Currently, this is done manually, and we intend to provide this as a service for the first customers. 

This map update model allows us to clear several certification hurdles while certifying that a visual global positioning system can meet availability and reliability criteria essential to use it either as a primary navigation system or as a dissimilar backup for a certified GNSS. 

Of course, for scalability and maintenance, it is highly desirable to automate the process. In our roadmap, we plan to do this and get rid of any manual supervision. The good news is that we know how to do this and are fully confident we will do it. In this future model, a VPS installed on aircraft constantly records the imagery obtained from its camera during flights; it extracts the features, inserts the newly found ones into the map and removes old features that have repeatedly failed to be recognised. Another piece of good news: we don’t expect any additional hardware upgrade to be necessary for this – the memory capacity of VPS is enough for storage of a large area.

An operator will enable fleet-wide sharing of the maps collected during normal operations, so that each of its aircraft can fly where any of its aircraft ever flew. Suppose a flight covers an area beyond current working VPS knowledge. In that case, the system can either pick up features localised against a working GNSS in real time or smoothly extend the known map by flying in an area where part of the known map is still visible, for example, at high altitude on a “reconnaissance mission”.

Whatever the mode of a map creation was, an operator can always see, both pre-flight and in-flight, the expected precision of the VPS work anywhere on Earth. Based on this, they can adapt flight plans and plan survey/reconnaissance flights accordingly.

Is this really the hill to die on?


All described above seems complicated compared to the GNSS navigation that comes, as seen by a user, without any effort. And, probably, the process is associated with costs (although we haven't yet decided on the commercial model for the maps). So, let’s take a look at the pros and cons of both systems.

GNSS receivers depend on its extensive infrastructure of satellites and ground stations. But the infrastructure base of any GNSS likely is not part of the certified unit (ILS will be different in this respect), and GNSS data is just assumed to be correct. Well... Let’s cast our minds back to 2019, when “the entire Galileo system suffered anunexpected and hitherto unexplained signal outage, beginning onJuly 11”.


U.S. Govt Source, Public domain, via Wikimedia Commons


Also, if we review the costs of running GNSS, they are not inconsiderable. Running the GPS infrastructure, for example, costs U.S. taxpayers over $2 million a day, not including the initial $12 billion to put the satellites into orbit in the first place. 

We expect the Visual Positioning System to compete with GNSS as a primary navigation system in both operational modes. For moments of GNSS outage during flight, the VPS has specific modes that keep it accurately referencing the local terrain even in the absence of recognisable features. So the sufficient quality of the signal is available for a ‘Land as soon as possible’ scenario. And if the recognisable VPS features are present, the mission can even continue as planned.