Four of the best professional poker players in the world spent most of January holed up at the Rivers Casino in Pittsburgh, losing. They’d show up before 11 am, wearing sweatpants and stylish sneakers, and sit down in front of computer screens. Each of them was supposed to play 1,500 hands of heads-up no limit Texas Hold ‘Em online before they could go back to the hotel for the night. This often meant working past 10 p.m. Over the course of the day, Starbucks cups and water bottles piled up next to the players's keyboards. Chipotle bags lay at their feet.
Every time one of the players made a move, the action was transmitted to a computer server sitting five miles away at Carnegie Mellon University. From there, a signal would travel another 12 miles to their opponent, a piece of software called Libratus running at the Pittsburgh Supercomputing Center in Monroeville, a nearby suburb. Libratus played eight hands at once — two against each opponent. It moved at a deliberate pace, slow enough to drive Jason Les, one of its human opponents, a bit mad. “It makes the days longer,” said Les, an earnest, athletic-looking man who seemed eager to take a few minutes off one afternoon last week. “Waiting should not affect me whatsoever, but sometimes you’re just like, 'OK, is this going to be over yet?'”
Libratus, of course, never needs a break. It's different from human players in other ways, too. People tend to think longer when there’s more money at stake. The computer plays most slowly on small pots, a result of having to scroll through all the additional possibilities that come from having more chips remaining in its hand. Libratus also tends to make huge, sudden wagers, violating standard betting conventions by throwing its money into the pot in irregular amounts and at odd intervals.
Coming from a human player, behavior like this would be irritating, reckless and, over the long run, expensive. But Libratus’s main attribute as a poker player is that it’s inhumanly good. When the 20-day tournament at Rivers came to an end Monday, the humans had lost $1.8 million. (They didn’t actually have to pony up the cash; money serves as the way of keeping score in poker.) Tuomas Sandholm and Noam Brown, the computer scientists at Carnegie Mellon who built Libratus, celebrated the win as the first time that a computer has beaten top poker players at a variant of unlimited Texas hold’em, the world’s most prominent poker game.
Experts in artificial intelligence have always used games as a way to develop and test their creations. Computers have surpassed the best human players at chess, checkers, backgammon, and go. Poker is a distinct challenge because of the element of chance, and because the players don’t know what cards their opponents are holding. So-called imperfect information games require the sort of human intelligence — like deceiving an opponent and sensing when she’s deceiving you— that computers lack.
“No limit hold’em is the game you see in tournaments, and it has the reputation of being more of an art than a science,” said Adam Kucharski, author of The Perfect Bet: How Science and Math Are Taking the Luck Out of Gambling. “There was the idea that this game would be safer for much longer from these machines.”
That idea has been blown up in recent weeks. In early January, researchers at the University of Alberta released a paper based on a contest in which their own AI, named DeepStack, beat 11 professional poker players.
Whether DeepStack beat Libratus to the punch is a matter of debate. Sandholm said that the pros who played against his bot were better than those DeepStack defeated. Michael Bowling, the head of the University of Alberta’s computer program, conceded this point. But he questioned whether humans are at their best when playing continuously for nearly a month. DeepStack's margin of victory was also three times that of Libratus's.
Both men agree that poker AI has just crossed a significant threshold. For them this has little to do with poker itself. Hold'em is just a way to find sparring partners for their artificial intelligence programs, and the gains made by game-playing bots will filter back into applications like cybersecurity. “This is the main benchmark the community has settled on, but these algorithms are not for poker,” said Sandholm, who was once one of the world's top-ranked windsurfers and kind of looks like Bill Gates. "They’re general purpose.”
DeepStack and Libratus play an unusual version of poker. The computers are matched up against a single opponent, as opposed to a group of players. The number of chips each player holds is reset after every hand, eliminating the complicated psychological game through which players with more chips intimidate poorer players by forcing them to make big bets. Eric Hollreiser, a spokesman for PokerStars, the world's leading online poker platform, said this limits any threat that AI poses to the poker industry. “While on a functional hand-by-hand basis it mimics poker play, it is far, far removed from the reality of what happens at tables,” he said.