We only have half of the Magic Leap SDK

Recently, Magic Leap released its SDK to developers so they can start playing with what is possible on the Magic Leap platform. With these tools you can develop a full Magic Leap application to run on the device when it is eventually released. But the documentation and toolkit is incomplete. There is a huge section that is still yet to be released to the public. 

That is, the Lumin Runtime. And it could completely change how you think about apps.

Recently, Magic Leap released its SDK to developers so they can start playing with what is possible on the Magic Leap platform. With these tools you can develop a full Magic Leap application to run on the device when it is eventually released. But the documentation and toolkit is incomplete. There is a huge section that is still yet to be released to the public. 

That is, the Lumin Runtime. And it could completely change how you think about apps.

Making an App

When you go to make an app on Android or iOS you have to choose the tools you want to use to make that application. There are 3 main choices: A Web App, using the official toolkit, or going native.

These days, web apps are so powerful that in most cases this is a good choice for developers. But if they need a bit more, they can use the official toolkits provided by the platform. This is using Java or Kotlin in Android and Objective-C or Swift on iOS. Most Apps are developed this way. But some apps, mostly games, need direct access to the hardware and need to more carefully manage compute resources. Platforms expose a specific SDK for this. On Android this is called the NDK or native development kit. 

What Magic Leap has released is the equivalent of Androids NDK. It has yet to release its application toolkit. This is important as there is a good chance that most applications made for Magic Leap will be written using this toolkit and not the 3D engines already released.

Lumin Runtime vs. 3D Engines

The Lumin Runtime sits beside the 3D engines in the Lumin OS stack. The SDK for these 3D engines are released already but we only have one short section on what the lumin runtime will be in the docs. It seems this runtime is geared towards developers, such as web developers, who might not be as comfortable working with game engines or starting from a blank canvas. It is for simple applications. For example, making an email client from scratch in unity or unreal is quite a daunting proposition for most developers. The Lumin Runtime helps developers by giving them a framework to use. It will provide a common design language for developers and help them navigate the challenging task of developing applications in an unknown 3D space. 

Throw out the App Model

One other detail about the Lumin runtime is that it is built for multiple applications to run at the same time stating: “It is capable of running and rendering multiple native applications simultaneously.” This is not true for the 3D engines which appear to take over the whole device. 

Traditionally, on mobile hardware, applications are installed and to use them you have to open them up and run them. One at a time. On android you have some more flexibility with this but generally most applications stick to this model. I believe (watch me wildly speculate again!) that Magic Leap will be a bit different. You won’t click a grid of icons to open an app but an app will live in a physical place somewhere. Let’s say you have a word processing app. It will live at your desk. To use it you can just walk up to your desk and it is there. A television app will live on the wall. You install it like a real TV. You might stick a mapping application to your front door so you can grab it on the way out. Perhaps have a recipe app beside your stove. The point is that you won’t need to scroll through your app drawer and find an icon to click on, the app will just be where you left it in physical space. This is all speculation on my part but I think it makes sense. 

The documentation says that the runtime will “facilitate application sharing out of the box, allowing application developers to collaborate with multiple users.” So if I install a TV app on the wall, everyone will see it. It is persistent and as part of the world as a real television. I could “hang” digital art on the wall and everyone who comes to my house would see that art through their own set of ML glasses. 

This is a radically new way to think about applications and computing in general. It is fundamentally tied to physical space and multi users. It truly is augmenting reality and I think one of the more exciting aspects of what Magic Leap is doing. 

How Magic Leap Works: SDK Edition

Some time ago I wrote about how I thought Magic Leap might work based primarily on patents that were published by the company. While patents do give some indication of what the company is working on, they do not reveal what features a product will have. At the time, that list of features didn’t exist. 

It does now. 

Some time ago I wrote about how I thought Magic Leap might work based primarily on patents that were published by the company. While patents do give some indication of what the company is working on, they do not reveal what features a product will have. At the time, that list of features didn’t exist. 

It does now. 

  • Audio: Stereo audio output and voice microphone recording are supported.
  • Camera: Capture still images and videos from the color camera.
  • Dispatch: Allows apps to open URLs using the Magic Leap Browser or other apps.
  • Eye Tracking: Ability to retrieve fixation point position and eye centers. Blinks can also be detected.
  • Graphics: OpenGL ES, Desktop, and Vulkan rendering paths.
  • Hand Gestures & Key Point Tracking: Recognize the user’s hand poses (gestures) and track the position of identifiable points on hands such as the tip of the index fingers.
  • Head Tracking: Headpose is tracked in full six degrees of freedom (DOF).
  • Image Tracking: Track the position and orientation of specified image targets in the user’s environment.
  • Input (Control / MLMA Support): Retrieve either 3 DOF (orientation) or full 6 DOF (position and orientation) from the Magic Leap Control. Detect button and touchpad presses and the analog trigger. Trigger values range from 0 to 1. A range of touchpad gestures are also supported, as are haptic vibration and LED ring feedback. This interface works seamlessly with both physical Magic Leap Controls and the Magic Leap Mobile App.
  • Light Tracking: Provides information (luminance, global color temperature) about the ambient light of the user’s environment.
  • Media Codec: Low-level, hardware-accelerated media encoding and decoding.
  • Media Player: Simple, straightforward media playback interface.
  • Meshing: Converts the world’s depth data into a connected triangle mesh that can be used for occlusion and physics.
  • Music Service: Supports connecting and listening to a streaming music service.
  • Occlusion: An interface for feeding depth data to the Magic Leap platform for hardware occlusion.
  • Planes: Recognize planar surfaces in the user’s environment for placing content. This includes semantic tagging for ceilings, floors, and walls.
  • Raycast: Fire a ray and get the point of intersection with the world’s depth data.
  • Secure Storage:Save data from your app to the device’s encrypted storage.

From this list, and other information in the developer documentation, we can make some better guesses at what the Magic Leap One will be capable of doing and what it will struggle with.

Eye Tracking

Depending on how well it works, eye tracking is going to be a big part of using Magic Leap. Without a screen to touch or a mouse to move, selecting things in Mixed Reality has always been challenging. Gestures are not as accurate as you might like and using headpose means a lot of head moving. Being able to just look at something and then either blink or do a small gesture will be a revolution in this space. As a real world example you can see what Eyefluence (a company recently purchased by Google) has done in this space. We don’t really know how it works but they don’t even need blinks to confirm actions. If this level of eye tracking is in the Magic Leap One then it will be heavily used for user interaction

Hand Gestures

Magic Leap One will support 8 hand gestures. This is notably more than Hololens. These gestures in concert with eye tracking will form the basis of interaction with magic leap. At least for applications that don’t use the controller

Control

The Magic Leap controller is fascinating. It has the basics you might expect: touchpad, home button, trigger, etc. But it also features 6dof tracking. We do not know how accurate this tracking will be but to provide orientation as well as position is quite challenging using accelerometers alone. As we have written about in the past, there are indications Magic Leap is using magnetic tracking to facilitate this. The protrusions that stick out the side of the lightwear might well house the magnetic coils necessary for this tracking. If this is the case, we may have tracking accuracy comparable to the Vive but without the need for clunky external cameras or lighthouses. 

Another thing revealed in the docs but not outlined in detail is the existence of a Magic Leap mobile application. Presumable for Android and iOS, this application can be used as a controller for Magic Leap. The documentation implies that it will transfer touch inputs from the phone to Magic Leap likely working like the trackpad on the controller. I think this might point to the controller being sold separately as you can use your smartphone if you don’t want to spend the extra money. It will be interesting to see what the mobile app will look like.

Occlusion/Depth Mapping

Occlusion is supported in Magic Leap directly in the API. Occlusion is a notoriously hard problem that no one has quite got right before in a consumer device. It requires accurate depth mapping that both project tango and hololens struggled with. That said, there are good examples of occlusion working in those devices for very specific circumstances. I suspect Magic Leap is doing some amount of guesswork, potentially using machine learning, to make better guesses at hard edges for occlusion. 

The documentation points to Magic Leap using IR based depth sensors similar to those used in Project Tango and Hololens. This means they will struggle to work in bright sunshine and on dark materials. 

The documentation makes it clear that this will not work for dynamic, moving objects. It states:

The environment is expected to mostly be static and change (such as objects moving or people walking in front of the device) only happens occasionally. For small changes, the reconstruction should slowly update over several seconds to add new objects or remove moved objects. Environments with significant or continuous change may lead to holes or incorrect geometry.

This means occlusion won’t work if someone steps into your field of view. Digital objects will appear in front of them. Hopefully, in future iterations this can be improved. 

Image Tracking

Image tracking is built directly into the Magic Leap SDK.  This allows Magic Leap to identify a given image and track its position in the real world. This isn’t a feature I was expecting to see at the API level. I can envision an application that takes artwork on your wall and bring it to life. Below is an example of this, though not running on Magic Leap. Warning, the audio in this video is intense.

ARCore/ARKit Parity

The SDK also includes features such as plane detection and light tracking. These are sort of table stakes for an AR toolkit at this point. 

Lumin OS

Magic Leap will run a custom operating system called Lumin OS. Lumin is based off of Android but heavily modified to support the requirements of spatial computing. This is good news for developers as it means the platform will be familiar for anyone with experience working with android or working with linux in general. It also means that Magic Leap is not trying to reinvent the wheel as it will leverage the years of work that has gone into the android open source project. 

One thing we can see in the listed features above is something called Dispatches. This sounds like a renaming of Androids Intent system. This is the system that allows you to select certain apps to use for different actions, like selecting a default browser. While it remains to be seen if Magic Leap will allow users to select default applications or if they will force users into their in house apps, this is a good indication that we might get more choice on this platform than some other notable platforms.

How well does it work?

The documentation gives us a very good idea of what to expect from Magic Leap. We pretty much know what this thing is going to be at this point and on the surface it looks amazing. No device has had this range of features in such a small form factor before. But the question that this does not answer is how well does it work. Is the controller tracking accurate? How is the image fidelity? Is eye tracking accurate? What kind of performance can we expect from the GPU/CPU? You can have all the features in the world, but if they perform poorly it won’t matter. We’ll have to wait for the device to come out later this year to find out.

This week is shaping up to be huge for Magic Leap

GDC is on us and it looks like Magic Leap is coming out in force. 

The team has been tweeting out images of a building decked out with Magic Leap logos and even a shot of a NASA spacesuit just in case we all forgot that TED talk from years ago. 

I expect they will be showing demos in that room to either the press or GDC attendees. Worst case it will only be for select developers but regardless there are going to be many more people that have experiencing Magic Leap by the end of the week. 

At the same time it appears WETA Gameshop is having a coming out party of sorts. They have been working on a game called Dr. Grordbort for over 6 years in tandem with the development of Magic Leap hardware. I suspect we will finally see in-game footage similar to the concept they released years ago. 

 An SDK release is almost certain. This was tweeted by Magic Leaps cloud architect.

 And as users on /r/magicleap discovered https://developer.magicleap.com has changed. Still inaccessible, but the error message is new and the portal below the login appears to be “opening”. 

 Should be a fun week ahead. I’m extremely curious how they decide to show footage of the device in action. We are going to know a lot more about what this first iteration of Magic Leap can do. 

 

It’s Here

Magic Leap has revealed their hardware for the world to see. It is at the same time surprisingly small and too large. It is brimming with sensors. It’s dense. Solid. It feels tightly packed. As if every square centimeter has been filled with circuitry. It’s somehow nostalgic. It’s the future but how the past envisions it. Cutting edge but at home in the ’80s.

It’s bulky. Too bulky. This isn’t for regular folk to walk down the street in. It looks like it is out of a movie. The too cool and aloof techy wears this while performing hacking miracles. But most importantly it is real and you can buy it in 2018.

It’s Here.

Today, the Magic Leap One form factor was revealed in an article by Rolling Stone and through Magic Leaps own website. And for the first time we have a ship date. Consumers will be able to buy the “Creators Edition” in 2018. This has been a long time coming. 

The Magic Leap One

The first product consists of three pieces: Lightwear which is the headset, the Lightpack which is a connected compute and battery unit and Control, a six degree of freedom input device.

Lightwear

This is it. The most important piece of Magic Leap One. The headset is the first impression of the device and the core of the technology. It is at the same time surprisingly small and too large. It is brimming with sensors. It’s dense. Solid. It feels tightly packed. As if every square centimeter has been filled with circuitry. It’s somehow nostalgic. It’s the future but how the past envisions it. Cutting edge but at home in the ’80s.

It’s bulky. Too bulky. This isn’t for regular folk to walk down the street in. It looks like it is out of a movie. The too cool and aloof techy wears this while performing hacking miracles. 

It looks like it could be comfortable. Rolling Stone agrees saying “The goggles were so comfortable you almost forget you’re wearing them”. They don’t look like they hang off your head like most headsets do. They are tight. Everything about this product looks tight. It looks lightweight. There are no extra straps. It doesn’t sit on your ears relying only on the bridge of your nose and the band around the head well clear of the ears. In fact, there don’t appear to be headphones at all. Rolling Stone states there are “Tiny, high-end speakers built into the temples of the device”. This isn’t private audio. This is a loud device to wear. It doesn’t fit into the background to be forgotten or ignored. Everything about it screams. 

It has 6 cameras. 4 microphones. 2 speakers. 2 wires coming out of the back. A compute unit for on headset, realtime processing. Eye tracking. Gesture recognition. And of course the two photonic wafers to transmit the lightfield into your eyes.

It is a first iteration product. This isn’t for everyone. It isn’t friendly enough for that. It has too much character for the mainstream. You can read that as it’s too ugly but I think that does it a disservice. In the right context this this looks cool. But it is for earlier adopters and developers. For folk with a lot of disposable income and a need to see the future today. 

Lightpack

The lightpack is huge. It is far bigger than I expected. This doesn’t fit in your pocket. It hangs out of it. It is a large, clip like design. One half goes in your pocket or belt  (or shoulder holster it seems?) and the other half sits outside of it. This isn’t just a smartphone. The space they have means they can do much more. I suspect a large chunk of it is battery. But even if half if it is battery they still have more room than a typical smartphone. It sounds like they aren’t using mobile hardware. The announcement is absent of specs but they talk about it like they are using hardware that is typically in laptops. This means the potential for far more horsepower than we see in an iPhone or the Nintendo Switch. 

As with the Lightwear, this isn’t trying to hide. It is out there. On your hip. For everyone to see. With a cable that splits and snakes up to your head, you will know you are wearing it. It will be exciting to discover exactly what the lightpack can do.

Control

Control is one of the many input modes for Magic Leap One. It contains an array of buttons, six-degrees of freedom motion sensing, haptics, and a touchpad.

It looks comfortable and benign. A similar yet simplified version of other controllers of this nature. The big question is how well the motion control works. Is this vive or oculus level accuracy? Is it simply driven by an IMU and prone to drift and recalibration? 

Combining this with hand tracking, head position, voice input and eye tracking we have many input modes for the Magic Leap One. We have to hope they have come up with some good interaction primitives to tie this all together in a intuitive and consistent way.

All Together Now

We finally know what Magic Leap has made and we know when it is going to be shipped. This is a huge moment for the company and will finally put to rest many of the more extreme doomsayers. Magic Leap is not Theranos. There is a real product. People have tried it and they can finally talk about it. 

What remains to be seen is if it is any good. This coming year will reveal all.

For the first time, Magic Leap is marketing to consumers

After near radio silence for years, Magic leap is finally talking to consumers. With a redesigned website, new logo, consistant messaging, and more active communications we are finally seeing the tone and the image that Magic Leap wants people to associate with their company.

For all the hubbub, Magic Leap has hardly said anything publicly about that they are doing. Discounting what has happened in the last week, they haven’t released a video for over a year. The last official post by them was in January. Most of what we hear about Magic Leap comes from funding rumours, hiring announcements, small and often misleading leaks, and the occasional post from folks under NDA. It is hard to argue that much of this communication was targeted at consumers. It was, for the most part, to help secure funding and to aid hiring.

That has all changed. They released two “stories” on their new website last week. Prior to this they had seven total. After a year long drought, they released a new video with the promise of part two coming soon. They have changed their logo across all media platforms. They have been actively responding on Twitter and even more so on Facebook. And of course they have completely redesigned their website

Their marketing has changed and they are finally starting to talk to consumers.

 Original Logo
Original Logo
 Old Logo
Old Logo
 New Logo
New Logo

This all points to a reveal coming soon


Their new website has a core focus on reintroducing themselves. They are offering mailing list that is the start of building a conversation with consumers. Unfortunately, the video they released along side this redesign is about as generic a marketing video as they come. It seems they want to start the conversation but are not quite ready to spill the beans on their technology yet. This puts the marketing team in a tough position. They want to build the tone of this company so when they do finally reveal what they have been working on, they have a brand identity to work with. 

The iconography they are using is consistent and strong. I imagine we see a lot more of this in the near future. This playful and colourful tone is the image that Magic Leap wants consumers to associate with them and for the first time, they want consumers to take notice.

A reveal is almost certainly just around the corner.

Magic Leap is looking for more money

So it appears Magic Leap is looking for even more money. This would be on top of the ~$1.4 billion already thrown at the company.  That amount is already unprecedented so talk of further investment is nothing short of astonishing. The thing that got me interested in the company in the first place was the ludicrous amount of investment they had raised from such interesting and respected sources. Piling more on top of this is just wild. 

So it appears Magic Leap is looking for even more money. This would be on top of the ~$1.4 billion already thrown at the company.  That amount is already unprecedented so talk of further investment is nothing short of astonishing. The thing that got me interested in the company in the first place was the ludicrous amount of investment they had raised from such interesting and respected sources. Piling more on top of this is just wild. 

If they are able to raise this money, this is great news for Magic Leap. They have already gone through a small PR firestorm last year when an article from The Information (paywall) cast doubt on some of the more ambitious speculation surrounding the company. Many people and even publications have held this up as evidence that Magic Leap is in big trouble.  This largely stems from the sort of cynicism and entitlement that is rife across the internet and does not accurately reflect the article in The Information. I have already spoken about my distaste for this towards the end of this article so I won’t bore you with it again. Regardless of why, and regardless of its baselessness, public sentiment is against the company. Given that climate and the amount already sunk into the company, a raise by Magic Leap at this point is all the more impressive and promising for their inevitable release.

Magic Leap reportedly had a demo to their board (who, as a reminder, are their investors) in February. While we don’t know what happened in that meeting, we can only assume it went well if they are willing to up more investment. You have to think the likes of Google and Alibaba would take a particularly critical eye at this stage and given the public pressure on the company.  Passing this tests speaks volumes. 

Why do they need more money?

More money means that Magic Leap must be doing something right but they would only do another raise if they needed more money.  What have they spent $1.4 billion on? That is a massive sum. Karl Guttag has rightly pointed out that Avegant has been able to develop similar technology to Magic Leap with significantly less investment. I don’t want to minimize what Avegant has accomplished as it sounds impressive but it is not a product. It is a prototype. Presumably it is a prototype similar to what Magic Leap has likely had for some time now, perhaps as far back as 2015.

Building a product, not a prototype, at scale is hard. And expensive. It is particular hard and expensive when you decide to build not only the product but the factory that makes the product. This is almost unheard of. Not even Apple exerts this sort of control over its supply chain. It becomes even harder and more expensive when you spearhead a large portion of the content creation in house. Only Nintendo can claim to do something similar. Meanwhile, you are still maintaining a massive R&D effort to continue to build on the first iteration of the product and think towards the future.  These sorts of R&D labs are what Google and Microsoft are known for. While all of this is ongoing, you have to engage with other companies to form a developer ecosystem (which appears to be happening behind closed doors rather than in the open) so that the content you produce is supplemented by 3rd parties. This is all expensive.

Magic Leap is trying to be the best of Apple, Google, Microsoft and Nintendo all in one package and they are trying to do it pre-revenue. So yes, they need more money. And yes, what Avegant did is impressive but that part of the product is the “cheap” part. Don’t confuse this with the easy part. It still requires a monumental engineering effort. But the expensive bit, the bit Magic Leap is currently in the middle of, is making that engineering effort into a compelling product people want to actually buy. This is why Magic Leap has spent its money and this is why Magic Leap is raising. It is damn fascinating that it looks like they might be getting what they ask for.

Mixed Reality mapping of the world will change the world

We all use Google Maps. It is arguably the most useful part of a smartphone. Being lost is, to a degree, a thing of the past. It is one of the most important innovations in recent memory and is perhaps the ‘killer feature’ of a smartphone. If Mixed Reality becomes popular, and we stretch our privacy a bit, then we are about to see another massive shift in mapping and how we see the world in general.

We all use Google Maps. It is arguably the most useful part of a smartphone. Being lost is, to a degree, a thing of the past. It is one of the most important innovations in recent memory and is perhaps the ‘killer feature’ of a smartphone. Over the years, it has been improved and Google, among others, have been able to more accurately map and model the world. While it is nice to take a moment to appreciate the importance of maps, I bring this up to show how far we have come with mapping technology. And then I want to say that what is coming is going to blow that out of the water. 

What does Mixed Reality have to do with mapping?

On the surface, a Mixed Reality project doesn’t seem like it has that much in common with mapping. Of course there will be mapping products built using the tools of mixed reality. Some of the first examples of this already exist as seen with a Tango device. While a this makes for a compelling product and has the potential to be the main way in which we experience maps or navigation, it isn’t the revolution I want to talk about.

Passable World Models

Something that Magic Leap has talked about since the first patents started to come out, is the idea of a Passable World Model. A Passable World Model is just what it sounds like, a model of the world that we are able to pass to others. When you want to display something in mixed reality you need to map out (or model) the space in high detail so as to know how to position digital images as if they are in that space. That mapping is often referred to as SLAM and it produces a set of data that allows a device to know its relative position in an area. It is needed on the device but there is nothing to say it could not be shared with another user. This means that the other users device does not have to go through the effort of mapping an area. It already has the data. Someone else has passed them that model of the world. This frees up processing for the task of displaying mixed reality content. 

While this technique is useful for mixed reality, its application goes beyond just that. We are talking about mapping spaces in extreme detail. The data could be used to create an accurate and relatively high fidelity 3D model of the space around you. Imagine a fleet of millions of users mapping out everywhere they go and uploading those maps to share with others. This information can potentially be used to reconstruct a virtual version of the world. Think of a video game but the world you inhabit is an accurate representation of the real one. Google Maps, and Google Earth, already do this to a degree but street view is clunky and 3D maps are fairly low fidelity when you get up close. Further those aspects of the product, while cool and interesting, are less useful than the more basic mapping functions we use every day. People don’t use a mapping app because it has nicer 3D maps. So while this new development would be cool, why would it be so revolutionary?

 Google Earths 3D looks really good.... 
Google Earths 3D looks really good…. 
 As long as you don't get too close
As long as you don’t get too close

Near Real Time 

If Magic Leap becomes popular, or any other device that will utilize SLAM mapping, then millions of people will be using this technology all the time, everywhere. A google maps like product could be constructed that uses that data to update maps in near real time.  Want to know if parking is available? If a place is particularly busy? If something is open or not? What the weather is like? Or just what the lay of the land looks like right now.

You could build a grand theft auto style game but use a real city in real time that potentially uses Mixed Reality to interact with people in the actual physical locations. Think pokemon go but with digital characters controlled by people at home mixed into the real world.  All these things might be possible.

The potential of this data is mind boggling. You could join a party from the other side of the world. You could work remotely but it really feel like you are in the office and going to lunch with your colleagues. If you take it to an extreme, something like this could slow urbanization. If being remote is even a half decent facsimile of being in a place in person then why wouldn’t I live in a cheaper area that is potentially more beautiful than a smoggy city. This is obviously a far off vision and the reasons for urbanization are many but one reason is certainly centralization for work. If people are just as effective working remotely as they are in person then that reason might dry up to a degree.  

Privacy

I’ve deliberately avoided talking about the privacy implications here. They are important but I think they are also so obvious that it almost isn’t worth talking about yet. I don’t think anyone imagining this would not come to the same conclusions about potential privacy violations with this sort of technology. Simple measures can be put into place. Having to opt in non public locations. Blurring faces as street view does. Perhaps it is possible to detect and omit people entirely from the mapping, perhaps you can even do that client side. We could do the same with cars.  Perhaps replace them with avatars so you can’t recognize the vehicle. It is a conversation to have but I don’t think it is a particularly interesting one at this moment in time. We keep changing the bar of what we think privacy should be so by the time we actually are able to build a product like this, our moral position on privacy may be vastly different to what it is now. The rate things are going today, perhaps no one will care about these sorts of privacy violations in the future. This may or may not be a good thing but it is impossible to deny the current trajectory of the last 25 years.

No more mystery: Magic Leaps lessons from recent launches

Magic Leaps next move is a product launch.  Given the press reaction to partial information, a “show don’t tell” mentality and a built in scepticism saying anything more at this point will be detrimental unless they can show it all. This product launch will be the most important moment for the company.  Obviously, their product needs to be great. Great products can overcome most obstacles. But not far behind is the messaging they are able to bring to the announcement. Many mediocre products have succeeded because of a good launch and many great products have failed due to a bad one. Whether it be a press conference, a pre-produced video or weeklong party, coming out of the launch announcement with praise from the zeitgeist can be a real difference maker.

There have been a few high profile product launches in the past year that have been fascinating to watch. Between No Mans Sky, The Google Pixel, New MacBooks and, most recently, the Nintendo Switch we have lots of fodder to go though and there is much for Magic Leap to learn. 

Magic Leaps next move is a product launch.  Given the press reaction to partial information, a “show don’t tell” mentality and a built in scepticism saying anything more at this point will be detrimental unless they can show it all. This product launch will be the most important moment for the company.  Obviously, their product needs to be great. Great products can overcome most obstacles. But not far behind is the messaging they are able to bring to the announcement. Many mediocre products have succeeded because of a good launch and many great products have failed due to a bad one. Whether it be a press conference, a pre-produced video or weeklong party, coming out of the launch announcement with praise from the zeitgeist can be a real difference maker.

There have been a few high profile product launches in the past year that have been fascinating to watch. Between No Mans Sky, The Google Pixel, New MacBooks and, most recently, the Nintendo Switch we have lots of fodder to go though and there is much for Magic Leap to learn. 

The New MacBooks

The 2016 MacBook launch is perhaps the least controversial and shows the most obvious points to learn. The biggest failing here was that the product was poor. Or perhaps not that the product was especially bad but that it was below expectations. New features were lacklustre and not practical while old features were lost. All the while the price was increased. Further the final product was buggy with persistent battery issues that were handle with an attempted subterfuge. This further eroded consumer confidence.  


A delayed game is eventually good, but a rushed game is forever bad.

-Shigeru Miyamoto


The biggest lesson here is you have to make sure you are releasing a great product.  As much as people want Magic Leap to launch soon, they should delay until the product is ready.

For Apple, it wasn’t just that the product was bad, it was that it missed expectations. Magic Leap is in a different position to Apple and does not have the weight or benefit of history behind it. Having a product that falls below expectations for them is not at all the same as for Apple. We don’t have a good idea of what our expectations are for Magic Leap while we have a solid idea of what they are for Apple.  This gives Magic Leap a window of opportunity with regards to managing expectations and is perhaps the single biggest lesson we can learn from all of these launches. People are greatly impacted by anchoring bias, much more so than most are willing to admit. Controlling that anchor, managing those expectations, can make a huge difference in the perception of the product.  

No Man’s Sky

From the least controversial directly into the most controversial. No Man’s Sky has, if nothing else, generated a lot of heated words on the internet.  I’ll say up front that I played the game and thought it was enjoyable. It wasn’t great. It wasn’t awful. It was… fine and, if played in a vacuum, you would think unassuming.  But, for better or worse, it was almost impossible to play this game without at least having some idea of the controversy around it. 

I do not want reiterate what the internet has already discussed at length so I will try to jump to the conclusion. No Man’s Sky is at the same time a great success and an abysmal failure.  For a ten man team, they sure made a lot of money. There is no doubt about the financial success of this product. But in making that money, they burned the brand. I don’t blame them for it and I certainly don’t think they did it on purpose though I know many are angry and do.  I think they genuinely wanted to make a fun and exciting game and were simply not able to put in all the features they dreamed and talked about. I would love the game that is inside of Sean Murray’s head, we probably all would, but it hard to manifest what is in your head.  

Frighteningly, Magic Leap has a lot of parallels with Hello Games and No Mans Sky. They both discuss their product in mysterious and abstract ways leading people to imagine grandeur where there may not be. They are both relative unknowns. As such, consumer expectations are not grounded in any previous product so the expectations are often wild, speculative and simply unachievable.  

While the failure of the latest MacBook launch was small, the backlash to No Mans Sky shows the extent of the damage that can be done when a company fails to manage expectation. Magic Leap needs to pop the hype bubble early. Luckily, and seemingly accidentally, this has already happened to a degree. Reed Albergotti article on The Information put a wet blanket on much of the excitement around Magic Leap and it has done it well before a public launch. As I said before, this is a blessing in disguise for Magic Leap and, if done purposefully which I would be doubtful of, an excellent lesson learned from the No Mans Sky launch.  

The Google Pixel

The Pixel launch looks like the opposite of No Mans Sky’s. While sales are likely going to meet Googles modest expectations they certainly aren’t going to break any records. For an iPhone or a Samsung phone it would be considered a commercial failure. But they did manage to successfully launch a brand, winning praise for the phone from most major sites, some of which rating it at the top of the smartphone market.  

This is the exact launch Magic Leap needs.  There is a good chance Magic Leap isn’t going to sell huge numbers of their first product but they have to show the world that the product is interesting. They have to convince people that even if they aren’t going to buy it, they do think it is cool. 

So how did Google do it? They took away surprise. The whole phone was leaked prior to the event. There was nothing about the device that people did not know going in.  They let the product speak for itself. They didn’t try to fool you with surprises, they stated what was on offer and let people try it for themselves.  Importantly, they didn’t lie or exaggerate features. Every phone company out there has said their camera is the best there is in the industry and they have all been wrong at some point or another.  Google said it, and according to many, the camera follows through.  Even if it isn’t the best it is close enough that people don’t feel lied to.  

Does this mean Magic Leap should take surprise out of the equation? No, I don’t think so. But the surprises have to be positive and they have to be true.  Don’t exaggerate battery life.  Don’t claim to have the best of something unless it is actually the best of something. To eliminate negative surprises, they should do controlled leaks prior to the event not containing the best of what they have but the things they think are the weakest. Get that out there early so people don’t get let down with inflated expectations. Then save the most impressive content for the surprise. 

Nintendo Switch

A lot of people have drawn parallels between Magic Leap and Apple in the past. Partially because Magic Leap have done that themselves.  But perhaps it is better to compare Magic Leap to Nintendo. They have many attributes in common, more so than with Apple. Nintendo has always been a strong R&D company and they have never been afraid to take risks. Sometimes those risks pay off and sometimes they fail but Nintendo is never afraid to try something crazy. There is no doubt what Magic Leap is trying is crazy.

Nintendo is also a content producer and deemed one of, if not the best in the business.  It appears Magic Leap is trying to follow that path. They are heavily investing in producing content in-house with hopes that the hardware they produce will actually have something that is worth running on it.  

This brings us to the launch of the Nintendo Switch.  A unique device boosted by some great exclusive content.  This is exactly what Magic Leap is hoping to release. Yet the launch wasn’t all roses for Nintendo.  Most publications spoke praise of both the hardware and software they were able to try but were hesitant to be too positive about the system. Nintendo had some unfortunate leaks prior to their launch event, the most damning of which was the price. It was widely leaked that the Switch would launch at $250.  Some were even hoping for $200. This turned out to be wrong, the cost is $300, and it framed the conversation around cost. If I was in Nintendo’s shoes prior to the event I would have leaked a price of $350. Perhaps even higher to ground people in the idea that this might be an expensive device. As it stands the cost is not high for the launch of a new system but the narrative has become that it is expensive.

For Magic Leap, they should leak a high price. Currently, no one knows what the cost should be for a product like this as few serious contenders have launched.  Magic Leap needs to anchor people in a high price. Cost for any product is relative.  The narrative of expense has almost as much to do with expectations as it does with what the product actually does. If people think it is going to be $1000 and it comes out at $700, then it will be thought of as reasonable.  If people think it is going to be $500 and it comes out at $700, then it will be thought of as expensive. Magic Leap needs to get ahead of this narrative and ensure that people think it will be more expensive than it turns out to be. By the way, just by reading this you have been anchored to a degree and your expectations have been affected. Keep in mind, we really don’t know how much this will cost. We don’t even know a ballpark.

Lessons Learned

In short, Magic Leaps PR department needs to do the following:

  • Leak your weakest material as early as you can and save your good stuff as a surprise. 
  • Leak a high price. Anchor people in that price.
  • Make sure your product is good. If it isn’t, delay. 
  • Leak. Control the narrative this way. You have already fostered an air of mystery so people will hang on every seemingly credible word. These leaks will define what people expect then you need to beat those expectations. Don’t be mysterious and let people build their own expectations which are too high to overcome.

Changing Expectations: What do we expect from Magic Leap now?

We were always hoping Magic Leap were keeping the cards close to their chest in terms of what technology they have developed.  It make sense. When the only people you are marketing to are investors and talent to hire then you want to make it seem like you have something that no one else does.  You want to make it feel like they could be part of something amazing. But that time has come to an end. They are no longer an R&D company. They are a product company or hope to be. They are no longer talking to investors and talent exclusively. They are talking to us. The consumer.

We were always hoping Magic Leap were keeping the cards close to their chest in terms of what technology they have developed.  It make sense. When the only people you are marketing to are investors and talent to hire then you want to make it seem like you have something that no one else does.  You want to make it feel like they could be part of something amazing. But that time has come to an end. They are no longer an R&D company. They are a product company or hope to be. They are no longer talking to investors and talent exclusively. They are talking to us. The consumer.

Consumers are more fickle.  They don’t like being lied to or having the perception of being lied to. They don’t like arrogance and hubris (unless you are Apple) and they don’t like engineering for engineering sake (unless you are Google). They like products. And they like products you can buy. 

 Consumers don't care and aren't excited by images like this.  We are past the point where fancy technology will sway us.
Consumers don’t care and aren’t excited by images like this.  We are past the point where fancy technology will sway us.

So, let’s look at what we expect to be able to buy given what we know. It has become clear that whatever secret sauce they have is far less exciting than was guessed at. That does not mean they don’t have something up their sleeve it just means it might not be as much of a leap forward as speculated. 

No big jumps in resolution

This was predicated on a novel display technology, the fiber scanning display, that does not appear to be ready.  A more traditional 1080p display is likely what we will get. The quote from reporters “There are no pixels” will not be true for the first product. 

No hugely increased field of view

Without the high resolution, Magic Leap will have a hard time increasing field of view by a huge margin.  They might have something interesting with regards to optics but the further you spread out the pixels the worse the image will look. Expecting something better than hololens but far worse than the Oculus or Vive seems reasonable.

Low fidelity variable optical focus

This is core to Magic Leaps philosophy. They need something here.  But there has been doubt cast on how well this will work.  Some speculate there will only be two planes of focus.  Others think it will all be done via eye tracking and software. This area is likely the most ripe for a surprise.  They might have something interesting.  The “photonic chip” is the only part of their product they have shown publicly so they are clearly proud of it. We should set our expectations low though as this is a hard problem particularly as the display technology they hoped to develop fell through.  

Inside out tracking on par or a bit better than tango

If there is one thing HoloLens should be proud of, it is inside out tracking. It does a great job at this. It is the best there is. We can hope magic leap is able to build something similar to this but likely they will fall sightly short.  This isn’t that damning though as you don’t need perfect tracking to make a useful product and it is something that can be improved as HoloLens has proved. I suspect we might get something similar to what google has achieved with project tango.  That is to say, we don’t get perfect tracking but it is passable. Objects appear relatively stationary with only some jitter.

With all of these downsides, you might question if Magic Leap has anything interesting to offer beyond what Microsoft and some others have built in this space. While they seem to be building something very similar to HoloLens I think there is one place where we will genuinely feel that they have built something far more interesting than the competition. 

Small form factor

We still don’t know what the form factor will be but all indications are that they are striving for something small, something glasses like. While no doubt they will be bulky compared to a regular pair of glasses they will likely look far more reasonable to wear when compared to the helmet like devices we typically see today. If Magic Leap is able to make a carbon copy of HoloLens but with a decent form factor and price then they will have achieved quite a feat.

 The ODG R-7 Smartglasses. These are something someone could easily wear outside without getting too many odd looks. 
The ODG R-7 Smartglasses. These are something someone could easily wear outside without getting too many odd looks. 

ODG was able to cram all the optics, battery, processing and a cameras need to build a display in to a pair of glasses.  It is missing inside out tracking which really limits their potential but the form factor they were able to achieve is promising.  If Magic Leap moves the battery and processing to the pocket unit then then they can use that conserved space to fit the cameras and technology needed for inside out tracking and potentially eye tracking.  

Should we still be excited? Should we still have faith?

If you were excited about cutting edge graphics that look just like real life then I’m sorry to say that isn’t going to happen.  If you thought that Magic Leap would replace all screens then you might have to wait a few iterations of the technology. As Rony Abovitz has said, the first product will be like the iPod to Apples iPhone.  It won’t do everything. It will be a bit bulky.  It might be a bit awkward to use.  But certain things it will be good at.  Certain core activities we do everyday on our smartphones will be better using these glasses.  Maps and Navigation are the first thing that comes to mind.  “Second Screening” when watching sports or television.  AR games that don’t require huge amount of horsepower (pokemon go style) will likely be a blast.  Table top games will be taken to a whole new level.  At work these could be used to have virtual white boards when people work from home and want to have meetings.  They could be used as a sort of second monitor that removes the bounds of the display.  

These are just examples off the top of my head.  None of them need high resolution. None of them need perfect tracking. None of them need high fidelity focal planes.  But all of them require us to feel comfortable wearing and using the product. We will never feel that way if we have to wear a giant helmet like device.  Form factor and a minimum level of tracking, graphics and interface are the most important things for Magic Leap to nail. I still have faith they can do that.

 

 

 

Magic Leap just got the best PR it could hope for

Magic Leap needed some negative press to temper these expectations and it needed it as far ahead of product launch as it could get. That pent up excitement, and resentment, needed to come out before the realities of the product crush the perfect image in our heads. 

Public sentiment is hard to grasp but somehow we can all feel it. Whether it be a company, artist or even about politics we get a sense of what the world or at least certain groups think about it. Very quickly a convergence of opinion happens and often that turns into what people believe to be true regardless of what the real truth is. This “truth” then becomes sticky. People cling to it. Sometimes they cling to it even in the face of overwhelming contrary evidence. 

For companies building a consumer product, being able to control this public sentiment, the zeitgeist, is vital. Yet very few are able to do it well or consistently. Magic Leap is amongst those that are struggling in this effort. You may think I am talking about the latest articles that have just come out but I am not. As I will argue, these articles are actually good for Magic Leap. I am referring to the uncontrolled expectations caused by the vacuum of information Magic Leap created. They created a mythical image of a product that was everything and nothing all at once. They created an imaginary device for people to project their wildest dreams onto.  No such device could truly exist. Those expectations could never be met. 

Magic Leap needed some negative press to temper these expectations and it needed it as far ahead of product launch as it could get. That pent up excitement, and resentment, needed to come out before the realities of the product crush the perfect image in our heads. 

Reed Albergotti’s article on the information confirmed much of what was suspected by Karl Guttag in his blog. Through first hand interviews and demos, Albergotti was able to show that some of the technology Magic Leap hoped to use was not yet ready to go in a consumer product. This means that some of the technical advantages that many assumed Magic Leap to have will not be seen in the first product. 

If this revelation was kept until launch day, the backlash would be monumental. There is a good chance it would sink the company regardless of how good the product is. But letting off some of the steam now, the product expectations will be lower. There will be more room for Magic Leap to make mistakes and more room for the things they get right to be brought to the forefront instead of being relegated as second fiddle to an imaginary perfect product.

So even if Magic Leap didn’t know it, even if they are upset about it, Albergotti’s article is exactly what Magic Leap needed if they have any hopes of succeeding in their first product. The hype train needed to be stopped. 

 The hype train needs to stop.  Only way to do that is to disappoint.  Better now, than later.
The hype train needs to stop.  Only way to do that is to disappoint.  Better now, than later.

Yet there is a fear that this negative press will create a public perception that will never go away. The stickiness of this new “truth”. I would argue that memory isn’t that long and products speak for themselves. By the time the product launches, expectations will be low and the negativity seen today will be a shadow of what it was. Most likely, people will be somewhat dismissive of the product but curious none the less. If the product is any good then they might be pleasantly surprised. If the product simply isn’t any good… Well then, it won’t matter either way. 

The PEQ

While petty ‘I told you so’s‘  and premature grave dancing continues on many sites, there is much to be hopeful for from the article on The Information.

This is the first time we have heard of the ‘Product Equivalent’ or PEQ.  This is a device manufactured at Magic Leaps factory that is close to the product they actually want to ship. While Albergotti did not get to touch or use the PEQ he did get to see it.  From his description, it proves that the most fundamental feature of Magic Leap is still a priority.  

Form factor

This is and has always been the most important aspect of Magic Leap. If they build a clone of hololens but it looks like a reasonable, if bulky, sized pair of glasses they will have built a compelling product. If they add some level of focal queues into that product, then it will still have a wow factor beyond hololens and way beyond what a typical consumer knows or expects. 

 From the information article:
From the information article: ” It looked as if somebody fastened electronics to every inch of a pair of wire-framed glasses”. Start with the wire frame and imagine some circuit boards.  Hopefully, the bulk added from that will be small enough that they still look reasonable.

The progress made on this sounds promising. They are really building it. We still don’t know what it will be capable of but we do know progress is being made. If nothing else, this negative press solidifies the transition Magic Leap has made from an R&D company to a Product company. And that product is coming. 

One last point 

Reading Albergotti’s article and you get a sense of both the issues Magic Leap is facing as well as the acknowledgement that they are working in a challenging space. You get the feeling that, yes, they aren’t what the hype implied but they are still interesting and worthy of our time. An impression to stay tuned because they still might make something great.   

The Verge on the other hand. 

After doing no research or reporting themselves, after being continually dismissive of the company in the past, they wrote a petty ‘I told you so’ article. It was oddly childish and doesn’t reflect what Albergotti wrote particularly accurately. 

I don’t get why we live to shoot down people who try something new and ambitious. Why we get this urge to say ‘No.  Stop. You can’t be good.’ Why we jump on them as soon as we see a chink in their armour and are proud of ourselves for it. Perhaps it is a form of humanities worst emotion, jealousy. Perhaps we feel they are belittling us because we aren’t trying hard enough so we fight against it. I sometimes feel this way and I hate that I do. 

We should be praising companies and people that try. Especially new companies that want to break the Google/Apple/Microsoft mould we are currently trapped in. We should celebrate their success and encourage them when they struggle. We should acknowledge that ambitious things are hard and not expect too much of them (something I am certainly guilty of). I enjoy much of what the verge writes but sometimes their hubris and their snarkyness gets the best of them. I hope they report with a more open mind in the future.