Tesla’s New AI Guru Could Help Its Cars Teach Themselves

Elon Musk has hired a new director of AI research at Tesla, and it may signal a plan to rethink the way its automated driving works.

This week, Musk poached Andrej Karpathy, an expert on vision, deep learning, and reinforcement learning, from OpenAI, a nonprofit that Musk and others are funding that’s dedicated to “discovering and enacting the path to safe artificial general intelligence.”

Karpathy, who will apparently report directly to Musk, is a rising star in the world of AI, having studied at Stanford with Fei-Fei Li, a leading AI expert who is now the chief scientist of Google Cloud. Li is famous in tech circles for having developed a data set of images that helped inspire a breakthrough in machine vision.

Many have pointed to Karpathy’s expertise in computer vision as a key asset for Tesla, and that’s true. But his experience with building reinforcement learning systems could be even more important for Tesla’s Autopilot effort.

Reinforcement learning is inspired by the way animals learn by repeating the behavior that led to a positive outcome. And, as we noted earlier this year, it has proved to be a powerful way to train computers to do things that are impossible to program (see “10 Breakthrough Technologies 2017: Reinforcement Learning”). This form of machine learning was at the heart of AlphaGo, a computer program from Alphabet’s DeepMind that learned to play the ancient board game Go with superhuman skill.

A number of carmakers, including Google, Uber, and Mobileye, which was recently acquired by Intel, are looking to reinforcement learning as a way to have cars figure out for themselves, in simulation, how to drive in challenging situations. Think of a four-way stop or a busy intersection. It is incredibly challenging to write the rules for a car to navigate such a situation. But, through reinforcement learning, it might be possible for a self-driving car to learn how to do this for itself.

After Stanford, Karpathy interned with DeepMind, where reinforcement learning is a major focus. The technique is also a big theme at OpenAI.

In fact, in a lengthy post about reinforcement learning on his own blog, Karpathy mentions reinforcement learning in the context of Tesla’s Autopilot. He notes that while reinforcement learning does not generally scale well to situations where experimentation is costly, new approaches, combined with lots of real-world data (of the kind Tesla is collecting) may help.

Appointing Karpathy a Tesla’s director of AI research indicates something else about the challenge of autonomous driving: there’s some distance left to go before it’s solved (see “What to Know Before You Get in a Self-Driving Car”).

(Read more: TechCrunch, “10 Breakthrough Technologies 2017: Reinforcement Learning”)