Wojciech Zaremba led a team that used machine learning to train a robot hand to teach itself to pick up a toy block in different environments. The robot was tasked with figuring out on its own how to accomplish the complex task of grasping a block and twisting it around with its robotic fingers in response to commands.

Zaremba powered the robot through a neural network, a computer program that mimics the type of networks our brains use.

Although reinforcement learning has been used before in robotics, it hasn’t worked on anything as complicated as a robotic hand, because the numerous tasks involved would require the equivalent of hundreds of years of experience. And robotic AIs trained in virtual worlds have typically failed to transfer successfully to reality, owing to the gap between simulated and real-world physics.

Zaremba, a cofounder of the AI research group OpenAI, hypothesized that varying the conditions in a virtual environment coiuld prepare a neural network for the messiness of reality.

He randomized 254 physical parameters—things like the mass of the block and the friction of fingertips—and found that the hand, after training, could manipulate the block the first time it was set loose in the real world.