New Tool Lets AI Learn to Do Almost Anything on a Computer

Machines may soon be trying to master just about anything you can do on a computer.

Open AI, a nonprofit dedicated to pursuing big advances in AI and making that progress freely available to anyone, has released Universe, a platform that will let AI programs learn, through experimentation and positive reward, how to do all sorts of things on a computer.

Universe will include more than a thousand games, but also desktop programs such as Web browsers. It will make it possible for AI researchers to train programs to do all sorts of new tricks, including potentially useful tasks like filling out online forms, responding to e-mails, and updating spreadsheets.

But Ilya Sutskevar, cofounder and research director at OpenAI, says the motivation for developing and releasing Universe is a lot bigger. Universe will provide way for AI researchers to develop and test algorithms capable of learning to perform a broad range of tasks—a step towards more general types of artificial intelligence. The hope is that it will lead to artificial agents that can learn a wide range of different tasks, and then take what they’ve learned in one setting and apply it to a different one. Such capabilities, known within the field as transfer learning, promise to increase the power and usefulness of artificial intelligence.

“If an agent does well on Universe tasks, which means it can understand what it needs to do, and do it as a result of applying its prior knowledge, then this agent will be significantly more intelligent than anything that exists today,” Sutskevar says.

AI algorithms can sometimes match or surpass human abilities, but only within very narrow domains, such as image recognition or playing a particular game. Most algorithms cannot learn to do lots of different tasks, and they generally cannot apply what they have learned in one domain to a different one.

The Universe environment lets AI agents take screen pixels as input and provide input in the form of keyboard strokes and mouse clicks. The platform will be compatible with AI agents that use reinforcement learning—that learn through experimentation and positive feedback. In the case of a computer game the feedback might be completing the game or finishing a level.

Sutskevar, lured away from Google by OpenAI last year, believes the platform will produce fundamental advances relevant to many different fields. “The most important product of Universe will be general-purpose algorithms able to learn from its vast experience across lots of domains and apply it to a new problem,” he says. “This algorithm could then be applied to robotics, natural language processing, and whatever else.”  

OpenAI was founded in December 2015 with a billion dollars contributed by big name technology investors including Elon Musk and Sam Altman, the president of Y Combinator, a well-known Silicon Valley startup accelerator. The nonprofit wants to push more AI research into the open as it increasingly becomes ensconced inside big tech companies.

Earlier today, researchers from Google DeepMind announced a platform to allow AI agents to learn how to interact with 3-D environments. This platform could conceivably be imported into Universe. Another game environment called Malmo—a version of Minecraft aimed at AI researchers—will be included with Universe.

Besides more than a thousand different games, Universe will include a large number of predefined browser tasks, such as filling out online forms or manipulating a Web calendar. This subset of tasks, called World of Bits, could inspire the creation of an AI that can do a large number of useful everyday chores, Sutskevar says.

Greg Brockman, another cofounder of OpenAI, says the nonprofit is releasing Universe not only to benefit the wider AI community, but also with the hope that others will expand its usefulness. “We think that by releasing it, people will help accelerate the platform,” he says.

Having others build even more tasks should help. “Success will look like a data set that is so rich and diverse that if you have an agent that does well across it, then it has to have learned something general.”

“It’s super-exciting,” says Emma Brunskill, a professor at Carnegie Mellon University who specializes in reinforcement learning. However, Brunskill, who focuses partly on developing AI systems for education, doubts that Universe will lead to a full-blown general AI. She says a key remaining problem is that it involves making lots of errors along the way, and it may not always be feasible for systems to experiment like this, especially if they need to interact with humans.

“All the domains [in Universe] are a little cordoned off from society,” she says. “In applications like education or health care, [agents] have to reason about the fact that they are high stakes.”

For example, Brunskill says, it wouldn’t be practical for an agent to figure out, through reinforcement learning, how to teach students math. “I don’t want to have a million students not learn fractions,” she says.