The Struggle to Spread the Minority Report Interface

In the 2002 film Minority Report, Tom Cruise’s cop of the future made use of a mind-blowing computer interface—a holographic wall of images and data floating before him, which Cruise manipulated by donning special gloves and making sweeping gestures to call up, move, zoom, combine, and discard far more information than fits on any PC screen, far more quickly. Few moviegoers realized this wasn’t a special-effects fantasy. It was a working system called gspeak, then under development by MIT Media Lab researcher John Underkoffler. Gspeak is captivating because it departs from the old desktop metaphor and mouse (and the currently hot tap-and-swipe touch screen) to let you handle things on the computer as if they were physical objects.

Public reaction to gspeak was so enthusiastic that Underkoffler was able to fund a company to turn his prototype into a real-world product. A decade later, his Oblong Industries operates out of a converted warehouse in downtown Los Angeles, where it builds high-end systems for deep-pocketed clients. He says he is picky about who he hires, because designing his product requires cross-disciplinary skills. “It works a lot better if instead of an artist and a technologist, you have people who are both,” he says.

Underkoffler says the military and companies wrestling with large-scale interactive data-visualization needs—such as financial institutions and oil and gas hunters—seek Oblong out. The system works—in fact, the next version won’t even require gloves.

So why isn’t it in wider use? Partly because a totally new type of user interface isn’t for everyone. “People don’t know what they want” in a UI, Underkoffler says. Sure, they’ll ask to have all four walls of a room covered in high-res displays, but it won’t occur to them that they should be able to grab a chart on one wall with their fingers, pull it away from the wall, and place it on the opposite wall as if it were a piece of paper. Instead, they’ll happily drag it all the way around the room the way they would in Windows. Focus groups and user feedback are helpful, he says, but they don’t lead to bold new design ideas.

Also, Oblong’s products remain too expensive to make the transition from awesome movie scene to the mass market. Oblong’s newest offering, Mezzanine, wraps a standard office conference room in high-res displays and a digital whiteboard. Meeting attendees can move things around with a wand that’s tracked in three dimensions. They point at something on one wall, click, and then pull back with their arm—you can’t do that with a laser pointer—to lift it from one screen and deposit it on another. Or they can operate Mezzanine less dramatically from their laptops and phones, which they can also use to upload presentations, pictures, and video. But a Mezzanine system is a pricey commitment (the company won’t confirm exactly how expensive), and it involves lots of jumbo screens and a rack of servers to process everything in real time.

Underkoffler expects that time will solve the price problem. Any consumer computer now has enough processing and graphics power to run the visual part of the interface. Microsoft’s Kinect game console offers gloveless motion control and costs $150. Still, Underkoffler dismisses comparisons of gspeak to today’s consumer computer interfaces. “People don’t realize the Xerox PARC demo that Steve Jobs adapted for the Mac was meant to be something a six-year-old could use,” he says. “Why should we as adults be restricted in what we can do?” Gspeak is more sophisticated, and more powerful, which sets up another hurdle to mass adoption: a learning curve.