Mixed Reality mapping of the world will change the world

We all use Google Maps. It is arguably the most useful part of a smartphone. Being lost is, to a degree, a thing of the past. It is one of the most important innovations in recent memory and is perhaps the ‘killer feature’ of a smartphone. If Mixed Reality becomes popular, and we stretch our privacy a bit, then we are about to see another massive shift in mapping and how we see the world in general.

We all use Google Maps. It is arguably the most useful part of a smartphone. Being lost is, to a degree, a thing of the past. It is one of the most important innovations in recent memory and is perhaps the ‘killer feature’ of a smartphone. Over the years, it has been improved and Google, among others, have been able to more accurately map and model the world. While it is nice to take a moment to appreciate the importance of maps, I bring this up to show how far we have come with mapping technology. And then I want to say that what is coming is going to blow that out of the water. 

What does Mixed Reality have to do with mapping?

On the surface, a Mixed Reality project doesn’t seem like it has that much in common with mapping. Of course there will be mapping products built using the tools of mixed reality. Some of the first examples of this already exist as seen with a Tango device. While a this makes for a compelling product and has the potential to be the main way in which we experience maps or navigation, it isn’t the revolution I want to talk about.

Passable World Models

Something that Magic Leap has talked about since the first patents started to come out, is the idea of a Passable World Model. A Passable World Model is just what it sounds like, a model of the world that we are able to pass to others. When you want to display something in mixed reality you need to map out (or model) the space in high detail so as to know how to position digital images as if they are in that space. That mapping is often referred to as SLAM and it produces a set of data that allows a device to know its relative position in an area. It is needed on the device but there is nothing to say it could not be shared with another user. This means that the other users device does not have to go through the effort of mapping an area. It already has the data. Someone else has passed them that model of the world. This frees up processing for the task of displaying mixed reality content. 

While this technique is useful for mixed reality, its application goes beyond just that. We are talking about mapping spaces in extreme detail. The data could be used to create an accurate and relatively high fidelity 3D model of the space around you. Imagine a fleet of millions of users mapping out everywhere they go and uploading those maps to share with others. This information can potentially be used to reconstruct a virtual version of the world. Think of a video game but the world you inhabit is an accurate representation of the real one. Google Maps, and Google Earth, already do this to a degree but street view is clunky and 3D maps are fairly low fidelity when you get up close. Further those aspects of the product, while cool and interesting, are less useful than the more basic mapping functions we use every day. People don’t use a mapping app because it has nicer 3D maps. So while this new development would be cool, why would it be so revolutionary?

 Google Earths 3D looks really good.... 
Google Earths 3D looks really good…. 
 As long as you don't get too close
As long as you don’t get too close

Near Real Time 

If Magic Leap becomes popular, or any other device that will utilize SLAM mapping, then millions of people will be using this technology all the time, everywhere. A google maps like product could be constructed that uses that data to update maps in near real time.  Want to know if parking is available? If a place is particularly busy? If something is open or not? What the weather is like? Or just what the lay of the land looks like right now.

You could build a grand theft auto style game but use a real city in real time that potentially uses Mixed Reality to interact with people in the actual physical locations. Think pokemon go but with digital characters controlled by people at home mixed into the real world.  All these things might be possible.

The potential of this data is mind boggling. You could join a party from the other side of the world. You could work remotely but it really feel like you are in the office and going to lunch with your colleagues. If you take it to an extreme, something like this could slow urbanization. If being remote is even a half decent facsimile of being in a place in person then why wouldn’t I live in a cheaper area that is potentially more beautiful than a smoggy city. This is obviously a far off vision and the reasons for urbanization are many but one reason is certainly centralization for work. If people are just as effective working remotely as they are in person then that reason might dry up to a degree.  


I’ve deliberately avoided talking about the privacy implications here. They are important but I think they are also so obvious that it almost isn’t worth talking about yet. I don’t think anyone imagining this would not come to the same conclusions about potential privacy violations with this sort of technology. Simple measures can be put into place. Having to opt in non public locations. Blurring faces as street view does. Perhaps it is possible to detect and omit people entirely from the mapping, perhaps you can even do that client side. We could do the same with cars.  Perhaps replace them with avatars so you can’t recognize the vehicle. It is a conversation to have but I don’t think it is a particularly interesting one at this moment in time. We keep changing the bar of what we think privacy should be so by the time we actually are able to build a product like this, our moral position on privacy may be vastly different to what it is now. The rate things are going today, perhaps no one will care about these sorts of privacy violations in the future. This may or may not be a good thing but it is impossible to deny the current trajectory of the last 25 years.

Leave a Reply

Your email address will not be published. Required fields are marked *