Brain Games

Marco Della Torre sits in front of a huge flat screen, wearing a strange, spiderlike contraption on his head. He slowly raises his arms, and a virtual rock begins to glow and shake on-screen. It falters a little and then rises, hanging briefly in the air.

Della Torre, a product engineer at San Francisco startup Emotiv Systems, is demonstrating the company’s new game controller–a headset incorporating sensors that can detect brain activity. Emotiv and its competitor, San Jose-based Neurosky, are developing the first gaming devices to use electroencephalography (EEG), a decades-old technology in which electrodes placed on the scalp measure electrical activity in the brain.

In the hands of neurologists, EEG can be a powerful tool for, say, identifying the source of seizures in epilepsy patients. But game developers want to use EEG to let players control virtual environments with their minds. They hope EEG will become the next big computer interface–a step beyond devices like the Nintendo Wii remote, which allow players to convert their hand movements into actions on-screen.

Both Emotiv and Neurosky have generated huge buzz in the gaming world. But can the headsets really provide the experience gamers are looking for? “People tend to want to do science fiction and [use EEG signals to] aim the gun or fly the plane,” says Scott Makeig, a neuroscientist at the University of California, San Diego. “But actually, changes in EEG are tied to changes in alertness and arousal that are slow by their nature. To try to use it to do fast things is not very natural.”

And measuring brain signals is tricky enough in the first place. Slight muscle movements, for instance, can generate electrical potentials more than 10 times as strong as those produced by neurons. Scientists who use EEG try to filter out this noise through various means: improving sensor sensitivity by applying conductive gel to the scalp, arraying sensors in caps specially fitted to a subject’s head, and employing complex signal analysis. “It has proved difficult to convincingly remove contamination from EEG signals,” says Alan Gevins, a neuroscientist and founder of SAM Technology in San Francisco, a company that is developing EEG-based medical tests for evaluating attention and memory.

In trying to design a headset suitable for gaming, Emotiv and Neurosky face a daunting challenge: it must not require sticky gels, it must be simple enough for any user to slip on (no matter how oddly shaped his or her head), and it must use sluggish EEG responses to control quick actions. To see how they’re faring, I flew to California earlier this year to test both their devices–with Gevins’s help.

Yoda-like
Sitting on a black leather couch at Emotiv’s sleek offices in downtown San Francisco, game producer Zachary Drake squeezes the Epoc headset onto my head. An outline of a head on the television screen in front of me indicates the quality of the signal coming from each of the headset’s 14 electrodes. Mine register mostly yellow–medium to poor.

I’m testing a game developed specifically to showcase the Epoc’s capabilities. In front of a pagoda set against a landscape of steep peaks and silhouetted trees, a martial­-arts master commands me to lift a rock using only my thoughts. Trying to summon a Yoda-like intensity, I focus on the rock and make a lifting motion with my hand. (While Della Torre says it’s not necessary to physically make the movement, it often helps.) At first, the rock does nothing. Then it lifts slightly, wavers in the air, and sinks.

The game includes a few training sessions, in which the software searches for specific patterns of electrical activity that occur when I either think about lifting a rock or make the motion to lift. The program will then try to detect and respond to those patterns once play begins. In theory, I don’t actually need to imagine lifting the rock: any specific pattern of brain activity, or potentially even of muscle activity, could be tied to a command. If I repeatedly thought about pushing during the training period, the computer would consider that the signal to lift.

That calibration of the system isn’t always precise. During my test run, the rock rises into the air as I turn away from the screen to jot in my notebook. I also blow up a mountain without meaning to do anything. Drake explains this away, saying I’m like a wizard just learning to cast spells–“They sometimes go off when you’re not even trying.”

The Emotiv headset has two other purported powers, both of which might be more in tune with EEG’s strengths. Using muscle signals picked up from face and eye movements, it can read facial expressions, so it could help animate people’s avatars–their digital alter egos–in virtual worlds like Second Life. (Software-enhanced cameras are under development to do similar things.) More novel is the headset’s ability to roughly detect your state of mind–whether you are engaged (be it angry or attentive) or not. The sky in the martial-arts game glows orange when I begin to concentrate and fades to green when I stare out the window.

Chris Linder, a developer working on a game that will ship with the commercial version of the Emotiv device, says that his team most enjoyed designing features that capitalize on these two aspects of it. While Linder, the cofounder of Demiurge Studios in Cambridge, MA, doesn’t want to divulge details of his nascent game, he gives the example of allowing the player to walk on water–but only while maintaining a constant level of calm. “If you got too excited, you would just sink,” says Linder, whose company also developed some of the first games for the Wii.

About 50 miles south of Emotiv’s offices in San Francisco are those of its main competitor, Neurosky, which has taken a much simpler approach to creating a commercial EEG device. The headset it has developed has just one sensor, which rests on the forehead, and a built-in processor that analyzes incoming signals before wirelessly sending data to a computer.

The Neurosky device attempts to detect different states of mind by analyzing brain waves–rhythmic fluctuations in the voltage measured by EEG electrodes. While there is no one-to-one correspondence between mental state and a specific brain rhythm, scientists have reported specific links. Certain patterns of theta rhythms, which occur at a frequency of four to seven hertz, are linked to drowsiness. Alpha waves, eight to twelve hertz, are characteristic of relaxation. (Alpha waves can also be induced by closing the eyes.) Algorithms distill the electrical signals registered by the sensor into a single output number, which can then be used as an element of control in a video game or other application. A second set of algorithms is designed to detect a mental state the company dubs “meditation.” (As was the case with Emotiv’s device, muscle activity probably also comes into play here. When you’re trying to concentrate, for example, you are likely to sit still and not move your head or eyes; the abatement of muscle activity may factor into the device’s analysis.)

Playing a simple game designed to display the device’s capabilities, I successfully lift a virtual block into the air by concentrating. I also set a branch on fire by sustaining that concentration: the branch sparks and smokes and then explodes into flame, an unexpectedly satisfying experience. Then, I gamely focus inwardly and manage to raise a sunken plane from a lake.

Caveat Emptor
What do neuroscientists–who have been using EEG for decades–think of these companies’ attempts to convert a research tool into a gaming technology? The ones I spoke with are uniformly skeptical that the devices rely on brain activity alone. (Neurosky owns up to this, while Emotiv’s president, Tan Le, insists that the telekinesis function uses only electrical signals emanating from the brain.) Gerwin Schalk, a research scientist at the Wadsworth Center in Albany, NY, is developing EEG-based systems that allow severely paralyzed people to interact with computers. He says that with his second-generation system, it takes several hours of training to control one degree of freedom of motion–what you need to lift a rock in the Emotiv video game. More extensive practice is needed to develop multidimensional control. “If you wanted to pick up signals to move a spaceship left or right, it would be much easier for a person to do it with facial expression than with brain activity,” says Schalk.

Alan Gevins, who has been studying EEG for the last 40 years, agrees. “If the sensors aren’t making good contact with the scalp, they will move slightly and generate an artifact when the head moves,” he says. “Such artifacts are often not that obvious and not easily removed algorithmically.” Emotiv declined to show us the raw EEG data collected by its device, citing proprietary concerns, so it was impossible to determine whether the headset and analysis software were truly filtering out noise and measuring brain activity consistently.

While Gevins acknowledges that for a gaming system, it doesn’t matter what kind of signals the device is using, he worries that overstating the ability of EEG to “read your mind” could damage the technology’s reputation. “They are way out on a limb with the labels they are putting on things,” he says.

Others hope that the EEG devices could have medical applications. Lesco Rogers, a pain management specialist at Duke University Medical Center in Durham, NC, has been in talks with Neurosky about testing its device for use with stroke patients. Rogers is considering very simple uses of the technology, such as allowing disabled patients to turn on a television. “What makes the technology interesting for me is the price point,” he says.

Meanwhile, EEG’s ability to measure alertness and arousal could add an interesting new layer to video games: in an unintended display of one of the Epoc’s features, the sky glowed bright orange as Della Torre, still wearing the headset after a demonstration, argued with a skeptical scientist. But the technology still seems too limited to have the transformative impact of the Wii. It’s true that Emotiv’s and Neurosky’s devices can, on a very simple level, read your mind–and lifting that plane with the powers of concentration felt very impressive. But the novelty of the devices is likely to wear off fast, and game players expecting the ability to exert precise mind control are likely to be disappointed.

Emily Singer is TR’s biotechnology and life sciences editor.