Magic Leap’s headset is real, but that may not be enough

Deep inside a nondescript building in Plantation, Florida, Magic Leap has built a gadget that is real, and cool, and can mix three-dimensional virtual images with reality better than any other augmented or mixed-reality headset—whatever you want to call it—that I’ve seen.

The big question now is: what will people do with this thing?

The company hopes developers and other creative types will start coming up with answers shortly. Because today Magic Leap will start selling its long-awaited first gadget, a pair of black, tinted, fly-eyed goggles called Magic Leap One.

It’s not for everyone. You’ll first have to register as a developer—the company hopes a community of developers will emerge to build apps for the headset, as they do for smartphones—and shell out $2,295 (for comparison, Microsoft’s HoloLens headset, also still aimed at developers, costs $3,000 or $5,000). You also have to be at least 18, and able to have it delivered to you in one of several US cities where it will be initially available, such as New York or Seattle. If none of these hurdles stops you, you’ll receive the headset, a wearable computer that connects to it, and a one-handed controller. A rechargeable battery gives the whole system enough juice to work for up to three hours at a time.

The ML One is the culmination of everything that CEO Rony Abovitz and the rest of his team have been working on since 2011. The company’s marketing, depicting things like a whale jumping out of a gym floor and a tiny elephant in a pair of cupped hands, envisioned a product so good at blending digital creations with the real world that it would kindle childlike wonder in users. The new headset represents a chance for Magic Leap to finally live up to the hype.

The news is that in many ways, it does. When I went to visit the company last month, I placed a headset over my face and saw sea turtles flying around the room in which I stood, leaving tiny trails of bubbles in their wake and moving when I poked them. I shot a ray gun at nasty alien robots as they jumped through a portal in a wall. Overall, the visuals were crisp and vivid, and in some cases I was able to see several digital images, positioned at various depths, at the same time.

It was way, way smaller and more portable than the early prototypes I saw back in late 2014 when I first visited the company. At that time, I looked through fixed lenses on a giant scaffolding-like device to see an impressively realistic blue monster. A cart-based (but still not portable) device let me view and poke at a tiny, steampunk-style flying robot. Even then, I was so impressed that MIT Technology Review included Magic Leap’s efforts—what it was then calling “cinematic reality”—in our 2015 list of 10 Breakthrough Technologies.

That was one of the few views outsiders have gotten into the company up until now. It has raised more than $2.3 billion, and filed for hundreds of patents related to things like projecting digital images in the real world (dozens of them have been granted). Yet for most of that time Magic Leap has combined maddening secrecy with exaggerated marketing stunts, leading many developers and techie consumers to conclude that the company was little more than vaporware.

In 2015, for instance, it released a YouTube video titled “Just another day in the office” that showed robots dropping into an office from a portal in the ceiling. The original description on YouTube stated it was “a game” the company was “playing around the office right now.” In fact, the clip was the same as or similar to one I had seen a few months earlier and had been told was simply a game trailer. (On YouTube, at least, where 4 million people have watched it, it has since been renamed “Original Concept Video”). Demos shot through ML One that the company showed off during a recent Twitch livestream, like a tiny rock-throwing monster, looked lame, and were mocked mercilessly.

So when I was asked this summer to go back to Florida to try the ML One ahead of its release, I was prepared for secrecy, obfuscation, and pretty lame visuals. I got some of the first two, but I think ML One is likely the best AR headset out there right now.

Yet while Magic Leap has accomplished what many people said it would not, it still has a monumental task ahead: convince developers to make compelling content for a style of computing that is so new that many people don’t know it exists, much less what kinds of things it will be good for. Figuring that out is not going to be easy. And my sense is that the company itself doesn’t have a clue what the answer is.

Since the last episode of “Magic Leap”…

Abovitz is tired, but cheerful. He stayed up until 2 a.m. last night working, trying to put the finishing touches on the ML One, which at the time I see him is coming out in just a few weeks. He’s sitting in his office, a glass-walled room in the middle of Magic Leap’s maze-like office in Plantation, Florida, about 10 miles inland from Fort Lauderdale’s palm-tree-dappled beaches.

Behind him are shelves filled with toys—everything from ray guns to a Jimi Hendrix figure to Miyazaki-like Ikea night lights—and books with titles like Making Hard Decisions and Graphics for Engineers. In one corner of a shelf, inside a small, clear case, stand figurines of pink and green monsters and an astronaut, along with a box of mythical Thwaxo’s Space Fudge—a small monument to Abovitz’s inscrutable 2012 stage appearance at a TedX event in Sarasota, Florida (the monsters were dancing around a giant fudge box, and he walked onstage in an astronaut suit).

He wants to give me a recap of everything that has happened at Magic Leap since we last spoke in person. If life were a TV show, my late-2014 visit was the pilot season, he says, and the company is in season four right now. He runs me through a slide show, pointing out years of prototypes ranging from the big one I saw (nicknamed “the bench”) to smaller and smaller versions (one wearable demo from 2015 looks like a Bladerunner prop and goes over the head, from the nape to the forehead). He wants to make it crystal clear: years have been spent building prototypes and fine-tuning different parts and pieces just to get to the ML One.

The company will not offer a clear explanation for how its headset blends digital images so nicely with the real world. Essentially, though, it’s shining light through see-through wafers built into the headset’s lenses, and those wafers direct the light toward your eyes. The way the company does this creates a digital simulacrum of the light field—all the light traveling in every direction in a certain volume—that you would see if the objects were actually in the space around you. With the ML One, Abovitz says, users should be able to see 3-D images clearly all the way from up close—the virtual light field starts 14.6 inches from your face—to out in the distance. Oh, and it should look totally natural, too.

After we’ve spent some time talking, we check out some of the many machines the company is using to build the aforementioned transparent wafers. Then, Abovitz walks me down the hall to a room with big, glossy white doors. It’s furnished like a hip living room, with a leather couch, side tables, bookcases, a large fabric-covered ottoman—the kinds of trappings that make a space look both meticulously set up and easy to imagine as part of your home, if your home is a Wayfair photo shoot set.

There is a table off to one side with some ML One headsets on it. Shanna De luliis, the company’s lead technical marketing manager, shows me the proper way to put on the headset: you need to gently pull apart the headband, place it over your head, and make sure it’s sitting at the right angle before tightening it. It is tinted, so donning it is kind of like wearing sunglasses indoors; Abovitz says the darker hue enables images to “look as solid-ish as possible” without making light from the product shine too brightly in your eyes. The headset isn’t meant to be worn with glasses (though you could probably fit them under there), but if you use corrective lenses the company says you can order a pop-in lens insert with your prescription strength.

Since I’m wearing a dress with no pockets, I must clip the round, connected Lightpack computer—basically, the processing and graphics brawn behind the operation—to a strap that goes over my shoulder. The headset, which Magic Leap also likes to call “Lightwear,” weighs 325 grams, while the Lightpack tips the scales at 415 grams; it’s like wearing heavy ski goggles while being wired to a fairly light purse. De Iuliis hands me a black controller with a touchpad on top and a trigger on the underside.

I try a series of demos, all of which will be either preloaded on the ML One or available on the company’s app store, Magic Leap World. The first demo is for a game called “Dr. Grordbort’s Invaders” that is in the works but will not be available initially on the ML One.

It starts with a rectangular portal opening in the wall in front of me. Yellow alien robots walk out, shooting at me. I dodge their fire and use the controller in my hand—which at this point has been overlaid with a digital image of a red, retro-looking ray gun—to shoot back. Every time I do, a smoldering black mark appears on whatever wall or piece of furniture I hit. It reminds me a lot of the misleading “game-we’re-playing-around-the-office” video from several years ago, only it’s an actual game I am playing around the office. And it looks nothing like the crappy, see-through demos Magic Leap has been criticized for showing on its Twitch live streams lately.

Whenever I shoot a robot, it falls to the ground in a heap. I stand over one and inspect its remains: the color isn’t bullet-proof, but it looks reasonably solid. At one point Abovitz had told me I’d need to really try it for myself to understand what he’s built, and he’s right: these robots (and most everything else I check out) look a heck of a lot better through the headset than on a flat display.

The company has done some impressive work in making digital images intermingle with reality, too. One of AR’s great challenges is making faux objects realistically block real ones, and vice versa—something that’s tricky to accomplish because you have to control light quite precisely. The demos I see do this well, mostly. Sometimes, robots emerge from behind a couch in the room (audio cues from speakers built into the headset help me sort out where they are if I can’t immediately spot them). However, when I try crouching behind a chair to see if they will disappear, the robot images sort of bend and come with me, rather than being properly occluded by the furniture.

Another demo, of an NBA app, lets me watch clips of a basketball game on a virtual flat screen, which I can move and adjust with the controller. I stick it to a wall. The image is reasonably clear unless I get up close (and about six inches away the image disappears, replaced by gridded lines). There’s also a “Court View” that shows me a tabletop-size 3-D model of a basketball court. I position it so it’s floating above the floor in front of me and watch a computer-generated LeBron James make a basket.

Yet another app, called Create, lets me build my own AR world with the headset and controller. I can grab little characters like a Tyrannosaurus Rex, a knight, or a sea turtle, and place them in my reality. I grab a T-Rex, who growls, and place him on an ottoman. He rolls over and then stands up, looking convincingly like he’s atop the taut fabric surface. I put a three-dimensional knight next to the T-Rex and the dinosaur smashes him flat.

I drag and drop sea turtles in midair, who swim away lazily. When I swat them, they move in response to my hand, and if I poke one with the controller I feel a little haptic zap. I let little bits of a tiny forest world sprout up on the furniture and the floor. I grab a virtual purple paintbrush and scribble all over Abovitz, who’s standing across the room, effectively blocking him out.

At that point, an error message pops up in my field of view. Apparently the headset can’t support any more creations, and I have to delete mine before making more.

“You crashed it!” Abovitz jokes.

There are still some glitches. While the visuals tend to look sharp and stay still when I swiftly shake my head, they sometimes split into red, green, and blue bits as I move around. Sam Miller, one of the founders and leader of the systems engineering team, tells me this is an issue of “tuning” the different pieces of hardware and software that work together.

More of an issue—not just for Magic Leap, but for every company that wants to mix real and digital imagery—is field of view. The ML One’s field of view measures 50 degrees at the diagonal, which is bigger than HoloLens’s 35 degrees but still only large enough to see, say, that virtual flat-screen TV from several feet away. Human binocular vision spans about 120 degrees, and more for each individual eye. VR headsets have a wider field of view than Magic Leap, but field of view is an easier technical challenge in VR than AR. Jeremy Bailenson, the founding director of Stanford University’s Virtual Human Interaction Lab, explains this is because VR typically involves placing displays in front of your face, while AR requires you to shine light and reflect it off a surface so it reaches the retina.

Tune in next time?

The excitement of the people I meet at Magic Leap is palpable, and it makes sense. Some of these folks have been working on this thing for years. Yet while the experiences in the demo room are fun and visually impressive, none of it is truly mind-blowing.

I want to feel like those robots are actually coming after me. I want the life-size whale coming out of the gym floor. And I want to forget that I’m weighed down by a headset and pocket-sized computer, peering at this visually enhanced world one rectangle at a time. For this to happen, the hardware will have to get still smaller and better.

More to the point, cool as the gadget is, the question remains: can Magic Leap turn it into a money-making business?

Abovitz has said in the past that the company’s first release would be a consumer product, not a beta device for developers as the ML One clearly is. Now, he says, “We realized we can’t bypass what I call creators, people that are makers. They’re first.” All the pieces of the puzzle—technology, hardware, and things to actually do with the headset—aren’t ready yet for most people to even consider buying it. Abovitz says it could take until the fourth generation of the headset to get to a popular gadget you would want to use all the time.

That is why the company is relying on developers and partners—a few big brands, including the NBA and Star Wars owner LucasFilm, are already on board—to help determine what will come next. But ask them what that might be, and Magic Leap’s leaders are curiously vague about it.

Rio Caraeff, the company’s chief content officer, tells me that the apps Magic Leap has been making for its test runs are more about activating your inner 12-year-old than “enterprise productivity or medical imaging.” Yet he also says the company wants to empower developers to “put their fingerprints on the world.” I hear a similar sentiment from Aleissia Laidacker, the interaction director: “I think once it’s out there we’re going to see these really amazing crazy things that developers are going to do,” she says. Abovitz, likewise, sells the let-a-thousand-flowers-bloom approach. “You don’t have to be, like a big company” to become a Magic Leap developer, he says. “You can be, like, a kid in a garage.”

Well, a kid in a garage who’s got $2,295 to burn.

This is the problem Magic Leap faces. Like the VR headsets on the market, it’s searching for its killer app (or  apps). Developers may discover amazing and useful things to do with this new computing platform, but they may not; VR developers have been failing to for years now.

Abovitz admits expectations for what he’s building have spiraled out of control, but he has no regrets. “You can’t have regrets,” he says simply.