Why Poker Is a Big Deal for Artificial Intelligence

As the great Kenny Rogers once said, a good gambler has to know when to hold ’em and know when to fold ’em. At the Rivers Casino in Pittsburgh this week, a computer program called Libratus may finally prove that computers can do this better than any human card player.

Libratus is playing thousands of games of heads-up, or two-player, no-limit Texas hold’em against several expert professional poker players. Now a little more than halfway through the 20-day contest, Libratus is up by almost $800,000 against its human opponents. So victory, while far from guaranteed, may well be in the cards.

A win for Libratus would be a huge achievement in artificial intelligence. Poker requires reasoning and intelligence that has proven difficult for machines to imitate. It is fundamentally different from checkers, chess, or Go, because an opponent’s hand remains hidden from view during play. In games of “imperfect information,” it is enormously complicated to figure out the ideal strategy given every possible approach your opponent may be taking. And no-limit Texas hold’em is especially challenging because an opponent could essentially bet any amount.

“Poker has been one of the hardest games for AI to crack,” says Andrew Ng, chief scientist at Baidu. “There is no single optimal move, but instead an AI player has to randomize its actions so as to make opponents uncertain when it is bluffing.”

Libratus was created by Tuomas Sandholm, a professor in the computer science department at CMU, and his graduate student Noam Brown. Sandholm, an expert on game theory and AI who emigrated from Finland for his PhD, says it is amazing that humans have been able to outplay computers for so long. “It just blows my mind how good these top pros are," he says. "Of all of these games that AI has tackled, [poker] is the only one where AI hasn't reached superhuman performance.”

AI researchers use game theory, or the mathematics of strategic decision making, to find the best strategy given various uncertainties, known as an equilibrium. Because the possibilities are so vast, this usually involves some form of approximation.

“Whether a move is good or not depends on things you cannot observe,” says Vincent Conitzer, a professor at Duke University who teaches AI and game theory. “This also results in a need to be unpredictable. If you never bluff, you are not a good player. If you always bluff, you are not a good player. Game theory tells you how to randomize your play in a way that is, in a sense, optimal.”

Last year, Sandholm led the development of a previous poker-playing program, called Claudico, which was soundly beaten in a match against several professional poker players. He explains that Libratus uses several new advances to achieve such a high level of play. This includes a new equilibrium approximation technique, Sandholm says, as well as several new methods for analyzing possible outcomes as cards are revealed at later stages of a game. This end-game analysis is computationally very challenging, and is performed during each game at the Pittsburgh Supercomputing Center, a facility operated by CMU and the University of Pittsburgh.

Advances in machine learning and AI have seen a number of superhuman game playing programs emerge recently. Last year, researchers at DeepMind, a subsidiary of Alphabet, developed a program capable of beating one of the world’s best Go players. This achievement was so spectacular because Go is extremely complex, and because it is hard to measure progress within the game (see “Google's AI Masters Go a Decade Earlier than Expected”).

A few different research groups are focused on tackling poker. Another academic team, from the University of Alberta in Canada, and Charles University and Czech Technical University in the Czech Republic, recently developed a program, called DeepStack, that has already beaten several professional players in heads-up no limit Texas hold’em (see “Poker Is the Latest Game to Fold Against AI”). However, Sandholm says, the players involved in the match against Libratus are far stronger, and are playing many more hands against the machine, which should provide greater statistical significance to the result.

The techniques used to build a smarter poker-bot could have many real-world applications. Game theory has already been applied to research on jamming attacks and cybersecurity, automated guidance for taxi service, and robot planning, says Sam Ganzfried, who was involved with the development of Claudico and is now an assistant professor at Florida International University in Miami.

However, even if Libratus triumphs this week, that doesn’t mean that humans no longer deserve a spot at the card table. The multiplayer version of no-limit Texas hold’em cannot be mastered using the techniques employed by Libratus.