Toyota Joins the Race for Self-Driving Cars with an Invisible Copilot

Toyota doesn’t just want its cars to drive themselves; it wants them to grab the wheel to stop you from crashing.

Toyota’s researchers are developing what they call a “guardian angel” system that will automatically take control of a vehicle, or subtly adjust a driver’s actions, in order to avert danger. In contrast to other companies working on self-driving vehicles, the Japanese carmaker sees combining machine and human driving as a key step toward full autonomy.

“In the same way that antilock braking and emergency braking work, there is a virtual driver that is trying to make sure you don’t have an accident by temporarily taking control from you,” explains Gill Pratt, CEO of the Toyota Research Institute, a company the carmaker created last year with $1 billion in funding to research automated driving, artificial intelligence, and robotics (see “Toyota’s Billion-Dollar Bet”).

Pratt announced the guardian-angel effort, as well as plans to create a new TRI facility close to the University of Michigan in Ann Arbor, during a speech at a conference in San Jose today.

Toyota’s approach raises new challenges, especially in terms of understanding and managing driver behavior. The company plans to test the technology in a giant moving simulator near Mt. Fuji in Japan. The simulator shows a “driver” realistic street scenes, moving around inside a hanger roughly the size of two football pitches. This will make it possible to see how people respond in realistic crash scenarios. “Our plan is to see how humans will respond when the car temporarily takes control because it knows better,” Pratt says. “So far the steering wheel always points in the direction the wheels go; that’s always been true up until now.”

The self-driving features on cars developed by other companies, including Google and Tesla, are either fully engaged or disabled. However, much existing safety technology, including power steering, lane-departure prevention, and automatic braking, are examples of partial autonomy.

A more gradual approach may have benefits because it can be difficult to transition from full autonomy back to driver control. Some experiments have shown that it can take eight seconds or more for drivers to regain their focus (see “Proceed with Caution Toward the Self Driving Car”). 

Pratt also suggested that Toyota will take a different computational approach. During his speech, he noted that existing self-driving cars use computers that consume thousands of watts. To achieve greater power efficiency, Pratt said Toyota could use neuromorphic chips, an architecture that computes data in parallel rather than sequentially, as conventional computers do.

TRI will hire around 50 people for the new institute in Ann Arbor. They will collaborate with University of Michigan researchers on self-driving cars and robotics.

TRI, which already has facilities in Palo Alto, California, and Cambridge, Massachusetts, will test prototype vehicles at all three locations, Pratt said. But the team in Ann Arbor will also use a specialized self-driving testing facility, called MCity, which can be used to mock up different scenarios (see “A Town Built for Driverless Cars”).

Although Toyota has been working on technologies related to self-driving cars for more than a decade, it lags behind Google and some other carmakers in terms of testing on real roads. This is significant because real driving data is needed to train the algorithms that control self-driving vehicles. However, Pratt says real-world training can be complemented by testing in virtual environments. He has previously said that Toyota’s self-driving cars will need to cover a trillion miles of road in testing before they can be used in the real world, and simulation may offer a way to achieve this.

Other companies, including Google, have talked about doing testing in simulation. And some academic researchers have shown the value of using highly realistic virtual settings to train self-driving algorithms (see “To Get Truly Smart, AI Might Need to Play More Video Games”).

Pratt and James Kuffner, TRI’s chief technical officer, said the ideal scenario would be for all the companies working on self-driving cars to share the data they are amassing—in both real and virtual testing—so that others can learn from it. “In the real spirit of safety—when it’s a public good—we strongly believe we should collaborate,” Pratt says.