Opposing evidence: Is Magic Leap actually boring?

If you are interested in Magic Leap (and if you are reading this site, you certainly are) then you really should be reading Karl Guttag’s blog. He has done some excellent analysis of the video that Magic Leap has released and come to some conclusions of what technology is being used. Unlike many reports, Guttag far less optimistic about the company. He paints a picture that puts Magic Leap in the same sort of place as many competitors. The advantages such as fiber scanning displays and high resolution are all but a dream in his analysis.

If you are interested in Magic Leap (and if you are reading this site, you certainly are) then you really should be reading Karl Guttag’s blog. He has done some excellent analysis of the video that Magic Leap has released and come to some conclusions of what technology is being used. Unlike many reports, Guttag far less optimistic about the company. He paints a picture that puts Magic Leap in the same sort of place as many competitors. The advantages such as fiber scanning displays and high resolution are all but a dream in his analysis.  Please do read the last few entries in his blog, they put forward some compelling evidence. 

 This looks good. But it doesn't quite line up with the quality of reports from the demo.
This looks good. But it doesn’t quite line up with the quality of reports from the demo.

On this blog, we have generally been optimistic about Magic Leap.  This is largely a result of all the reasons it makes sense to have some amount of faith in Magic leap but is further strengthened from the sort of philosophical idea that optimism is positive for the community (and the world at large). Unfortunately, reality has its own ideas and they often do not align with the optimist. Nor does it align with anyone else for that matter.  

How do we rectify analysis done here on GPU of the Brain and elsewhere with Guttag’s in depth analysis? Well, the first thing to note is Guttag is taking first hand information.  He is looking directly at the videos that Magic Leap has released. So we cannot chalk his ideas up to cold pessimism. Likewise, we cannot discount the reactions to demos seen by a number of independent people. These people are generally respected tech journalists, some of whom have been in the industry for years. These reactions show that Magic Leap must have something up their sleeve that is beyond todays tech.

As with most things, I think the answer lies somewhere in the middle.

Backlash

 Hololens doesn't have the field of view we were initially led to believe.
Hololens doesn’t have the field of view we were initially led to believe.

Magic Leap should be terrified of backlash to their product. It happened to Google Glass. It happened to Hololens and it will likely happen to Magic Leap. With this in mind, I suspect the videos that are being shown by Magic Leap are a projection of what they imagine their product will do at a relative minimum.  They don’t want to show a huge field of view or high resolution as hololens did only to later reveal that that isn’t the reality of the product.  They want these videos to line up as closely as they can with whatever it is they eventually come to sell. 

I don’t think Magic Leap have done a great job at managing the hype train but some of what Guttag has seen can likely be chalked up as an attempt to do this. 

“There are no pixels”

This has been said repeatedly by those who have tried the Magic Leap demo. As Guttag points out, in the videos released by Magic Leap, pixels are very clear. I don’t believe these reporters are lying but I think they are being shown a prototype that goes beyond the capabilities of miniaturisation and manufacturing at this point in time.  As I said in my previous piece on these demos, the most important question that can be asked about Magic Leap right now is: “Can Magic Leap transition this impressive demo into a form factor and product suitable for the consumer market?”.  The answer is unclear at this point.

Then what is the product?

Rony Abovitz has said in some recent talks that he imagines Magic Leap on a path that mirrors the iPhone.  He talks about the iPod being the first stop. If you go back and look at the original iPod you see a compelling product that is marginally better than competing devices of the time. That margin was enough for it to win the space. I don’t think Apple had a grand plan at this stage of the company but you can see the steps and iterative improvements as technology caught up to allow for a device like the iPhone.   

Magic Leaps first product will likely start the company on a path that technology will eventually catch up with. There is a good chance we won’t get a fiber scanning display in iteration one. We might not get a great FoV.  We might only get a glimpse of accommodation and vergence. But these ideas look like they hold water and five to ten years from now, we might get exactly what the larger demos are promising. They have the blueprints, they just need the time to build it. My biggest fear is that the inevitable backlash will kill them before they are able to truly make what they believe in. We should think about this when the device is launched. We should try to be optimistic. I know grave dancing is fun but it ultimately penalises companies for trying something new. We should be encouraging these companies not ridiculing them.

The people who have tried Magic Leap

At this point “1000s” of people have tried some incarnation of Magic Leap. Most of these people will be the employees or potential employees of Magic Leap, investors and content creators. All of them are under NDA and are not particularly interested in discussing their experience. But there are a few reporters who have been invited to try prototypes at different stages of development. They are still under NDA but are doing their best to tell us what they saw under these heavy restrictions. With each new article we see a slightly different twist on the bending of these restrictions and by putting them together we can build a more cohesive picture. 

At this point “1000s” of people have tried some incarnation of Magic Leap. Most of these people will be the employees or potential employees of Magic Leap, investors and content creators. All of them are under NDA and are not particularly interested in discussing their experience. But there are a few reporters who have been invited to try prototypes at different stages of development. They are still under NDA but are doing their best to tell us what they saw under these heavy restrictions. With each new article we see a slightly different twist on the bending of these restrictions and by putting them together we can build a more cohesive picture. 

To be clear up front, these first hand accounts paint the prototypes in a highly positive light. All accounts depict a demo that has blown them away and in some cases elicits a highly emotional response.  So who are these people?

Rachel Metz, MIT Review, Late 2014

I believe this article in the MIT Technology Review is the first time a reporter was given a chance to see and write about Magic Leaps technology.  In late 2014, Metz looked through “a pair of lenses attached to what looks like metal scaffolding that towers over [her] head and contains a bunch of electronics and lenses”.  This early proof of concept prototype is about as far from a consumer release as you can get but should represent, at the very least, what Magic Leap is striving for in a smaller form factor.

Metz was clearly impressed with the high degree of realism seen in the prototypes.  We will see this impression become a prevailing theme amongst people who have tried Magic Leap. She says, “I know there isn’t a hulking four-armed, twisty-horned blue monster clomping in circles in front of me, but it sure as hell looks like it.” Further in the article she states:


“I extend my hand to give him a base to walk on, and I swear I feel a tingling in my palm in expectation of his little feet pressing into it”


As we will see, this phantom sensation is reported by others when experiencing the technology. The level or realism needed to achieve this is obviously high.  It is perhaps the most impressive statement people consistently make about the technology.

It is not all roses though. Metz is quick to remind us that the prototype used to produce this impressive effect is a large piece of machinery further stating “The smallest demo hardware I’ve seen at Magic Leap can’t yet match the experience of the bigger demo units” and that it produced a “crude green version of the same four-armed monster”. The state of things in late 2014 seems to be an excellent proof of concept but a huge challenge for miniaturisation.

Kevin Kelly, Wired, late 2015


“[You] see two staffers playing tag on the table. A scale shift that made me feel like ‘honey I shrunk the kids’. … They were about 3 inches tall. … Your nose is in the guys hair. There are no pixels”


In early 2016, Wired did a large cover story on Magic Leap.  This is perhaps the most widely read piece on the company. Along with Kevin Kelly, Billy Sorrentino and Scott Dadich took part in the demo. The demo itself seems like it was a modest upgrade to the demo seen by Metz but still did not show consumer ready hardware. When the story was published, Wired released a podcast episode that talks exclusively about the demo experience. This podcast has more details about the specific demos than the article which talks in broader generalities.  Similar to Metz, Sorrentino and Dadich were both blown away by the technology stating “I was as excited as anything that I’ve ever seen” and “I did not sleep that first night back [from the demo]”. Others in conversation with them say, “Your excitement was thrumming around you, it was crazy”.  

The most common theme from these experiences is that “There are no pixels”. This is mentioned in the article, stating “pixels disappear”, and in the podcast repeatedly.  We also get a reiteration of that phantom sense that Metz felt, “you almost feel a sensation of tingling as the robot flies up to [your finger]”.  All of this points to a resolution far greater than anything we’ve seen in other devices. We get an indication that Magic Leap handles creating Black in stride “The eyeballs of the robot are black.  If you are projecting something you can’t project black, black is the absence of light.  Your mind is filling in the blanks. Your brain is sending is sending you that black signal.” Not only can black be produced but environmental reflections were seen in the demo, though, it must be said, not in all cases.  


“Very easily you can imagine a world where TVs are just mapped on your wall … It felt just like watching a movie in a movie theater. The only difference was absent the constraint the size of the TV … It felt like sitting courtside [at a basketball game]”


But, again, we get the same caveats. “There isn’t a device yet” just a  “headset goggly thing”. The demo “was like sitting in an optometrists chair”.  That said the ergonomics must have improved somewhat as they claim that “removing the Magic Leap’s optics was effortless, as comfortable as slipping off sunglasses, which I also did not experience in other systems. It felt natural.”  We also get an indication of how realistic the virtual objects seemed. While impressive they were not to be mistaken for real objects.  “It looks synthetic, it looks like a projected object … what is absent is the screen door affect”  and “They look like glowing objects, ghostbustery. Critters in avatar, glowing.” So all the excitement around the realism of the objects seen shouldn’t be take to completely fool you.  You will know what is digital and what is real.

David Ewalt, Forbes, Fall 2016


“The high-definition television hanging on the wall seems perfectly normal. Until it vanishes. A moment later it reappears in the middle of the room. Incredibly, it is now levitating in midair. Get as close as you’d like, check it out from different angles. It’s 80 inches diagonal, tuned to ESPN–and there is nothing holding it up.”


Just recently, David Ewalt got a tour of the Magic Leap offices, their fledgeling manufacturing facility and a full demo.  He wrote an article for Forbes outlining his experience as well as an AMA on reddit. He comes away equally impressed as others stating that Magic Leap is “changing computing forever”.  It sounds like his demo was closer to a consumer product but still not quite there.  He talks about seeing objects walking down halls, in lounges and even outside in a parking lot.  This implies a mobility that previous demos were not privy to.  He calls what he experienced “a head-mounted display, but the final product should fit into a pair of spectacles”. 

Ewalt seemed particularly impressed with Magic Leap as a replacement for a Television. In the AMA he states: “I can absolutely see a pair of Magic Leap glasses replacing my TV and monitors”. That is a bold statement and a vast industry to disrupt.  

Google has made a big bet on AI assistance and it looks like Magic Leap is too.  They showed Ewalt “computer-generated ‘virtual interactive human’, life-size and surprisingly realistic” where magic leap “[imagines] virtual people (or animals or anything else) as digital assistants”.  Given Magic Leaps connection to Google, I would speculate these assistants may be partially powered by the Google Assistant in the background. 


“Ask your virtual assistant to deliver a message to a coworker and it might walk out of your office, reappear beside your colleague’s desk via his or her own MR headset and deliver the message in person.”


The AMA was more about specifications of what the technology could do.  Due to an agreement with the company, Ewalt has to watch what he says but he has a few carefully chosen things to say regarding field of view. “I could get very close to MR objects without experiencing any problems.”  This is one of the main drawbacks of Hololens. When you get close to an MR object it fills more of your field of view and will be clipped if the FOV is poor.  This statement goes along way to say that clipping is not a major issue for Magic Leap.  Further he states,  “There were certainly times when an experience was playing out across my view, and things looked like they were happening all around me… it’s surprisingly immersive”.  When specifically referencing Hololens, Ewalt says: “I can tell you the FOV is impressively large, and bigger than HoloLens”.  

The immersiveness and realism expressed by Ewalt was even more hypebolic than we have seen from others.  “I can imagine a scenario where someone is completely fooled into thinking an MR object created by Magic Leap’s technology is 100% real”.  This is a strong statement.

Ewalt was shown the manufacturing facility Magic Leap is building in Florida.  He was shown “a series of long, self-contained, modular bays” that have the ability to “[ramp] up production from thousands of units a year to more than a million.” Magic Leap wants to be as agile as possible on hardware and given the challenge of manufacturing this new technology this is a must. I am encouraged by the concept of modularity and self contained components to facilitate this rapid and ever changing process.

Putting it together

Those who have tried Magic Leap have not been able to say much. What they have said can only be viewed as overwhelmingly positive.  The most consistent thing we have heard from all these sources is just how impressive and transformative the demo is. After reading these impressions, one cannot reasonably think Magic Leap isn’t on to something interesting.

 TV's floating in the middle of the room
TV’s floating in the middle of the room

But we shouldn’t discount the skeptic either. One thing is still not clear from Magic Leap or these lucky folks who have tried the prototypes and puts a dark cloud over everything said.  Can Magic Leap transition this impressive demo into a form factor and product suitable for the consumer market? We just don’t know and these eye witness reports don’t give us any information in that regard.  The best that can be said is they do have an impressive looking manufacturing facility up and running currently.  What is coming out the other side of these “submarines” like pods? Only time will tell.

EDIT, November 16th, 2016:

Further accounts

There have been a number of other people who have tried Magic Leap technology over the years.  By all accounts, they are just as overwhelmed by the technology.  Felecia Hatcher posted about his experience on instagram.  He claims “Your iPhones, your smart phones, all that stuff just throw it out. Your television throw that out too.”  You can see his account here.

We get further accounts from John Markoff at about the 24 minute mark on the all tech considered podcast.  He visited Magic leap in early 2014.  At the time he says that the technology was very much incomplete.  Even in this prototype stage he says, the resolution they had achieved was as good or better than any HD TV he had ever seen. We get another account of the brain being tricked by the high level of realism achieved in the demo. When Markoff put his thumb “inside” of the digital monster, his thumb disappeared.  We are lead to believe this was not the monster rendering over his thumb but his brain actually occluding it on its own.  You can listen to his account here