In a crowded convention center in San Jose, Calif., this past January during the Genesis 4 Super Smash Bros. tournament, away from the main competitive stage, a small group of gamers gathered around a clunky, four-year-old HP laptop. Amidst the onlookers, a professional player called Gravy was battling on familiar ground against an unfamiliar opponent.
The arena was Battlefield, a flat stage with three small platforms, considered the standard for professional play. He’s played professionally as Captain Falcon for nearly five years, and considered one of the world’s top players for the character—but he was losing to the AI playing as the same character. It had only been practicing for two weeks.
The AI, nicknamed Phillip, had been built by a Ph.D student from MIT, with help from a friend at New York University, and it honed its craft inside an MIT supercomputer. By the time Gravy stopped playing, the bot had killed him eight times, compared to his five kills.
“The AI is definitely godlike,” Gravy, whose real name is Dustin White, told Quartz. “I am not sure if anyone could beat it.”
AI has already beaten world class players at chess, poker, and Go—games that have nearly limitless permutations and require strategy. Super Smash Bros. Melee might be the most overtly adversarial of the bunch. Players try to gain advantageous ground while punishing their opponents, until they’re weak enough to knock off the stage. It requires strategic thinking and a certain level of viciousness.
But the bot was once only as good as a mere mortal. At first, Vlad Firoiu, creator and a competitive Smash player himself, couldn’t train Phillip to be as strong as the in-game bot, which he says even the worst players can beat fairly easily. Firoiu’s solution? He started making the bot play itself over and over again, slowly learning which techniques fail and which succeed, called reinforcement learning. Then, he left it alone.
“I just sort of forgot about it for a week,” said Firoiu, who coauthored an unreviewed paper with William F. Whitney, the NYU student, on the work. “A week later I looked at it and I was just like, ‘Oh my gosh.’ I tried playing it and I couldn’t beat it.”
By the time Genesis 4 rolled around, which Firoiu had entered anyway, he had a bot that played unlike any human. Akin to its AI-counterpart AlphaGo, from Alphabet’s DeepMind, the bot had learned its play style by battling itself. As a result, it behaved oddly. It stomped its opponents to their death over the stage’s edge, while a human player would typically take the safer route of letting their opponents fall. If a player tried to recover after the first stomp, it would stomp twice. This is an advanced form of edge-guarding, or a technique to prevent a opponent from recovering after a fall. Humans do it, but according to Firoiu , the bot is a jerk about it, rapidly spiking opponents when a courteous human would let gravity do its work (and have the safety of still being on the stage).
Gravy says the bot would dash dance, a human-credited technique of quickly skirting back and forth on the stage, keeping your opponents guessing about how you’ll attack. As for overall strategies, Gravy had some pointers. He suggested flow-charting, making a graphical chart of tried-and-true combinations of moves that work well when stacked.
“I asked the creator if he had flow charted the AI but was told that he didn’t, and that it learned its own strategies,” he said. “I do think it would be stronger to program optimal flow charts in.”
The bot almost learns to make its own flow chart. Based on its past playing experiences, it learns that certain combinations of moves are more effective, through thousands of games of trial and error. However, its preferred move combinations are strange, and almost inhuman to pros who watch.
It often uses a slower but more powerful move (a forward Smash) that’s rarely seen in tournament play because it typically leaves a player unguarded for too long. The bot also presents itself as off-guard or vulnerable, potentially trying to lull its opponent into striking while it actually had an advantage.
But when considering how the bot learned to play, it begins to make sense. The typical human has a response time of about 200 milliseconds, about six times slower than the bot’s 33 ms typical reaction. When learning against itself, the bot optimized for a quicker opponent than any human, meaning any human competitor moves like molasses in the bot’s world. Firoiu calls this alternate reality the bot inhabits a “meta-game.”
The researchers’ choice of character for the bot for competition, Captain Falcon, was intentional. He’s ”the worst character that can win a tournament,” Firoiu says, mainly due to the fact that he’s slower to execute moves than most of the top-tier characters. The team figured this would cut down on the bot’s reaction-time advantage. Captain Falcon is also one of the only characters that doesn’t fire any projectiles, which the team’s system can’t process.
But since the bot had only trained against itself as Captain Falcon, it played slightly worse against any other character. One professional player also found a glitch in the bot by doing something unexpected: By crouching close to the corner of the stage, the bot would not attack and eventually fall off the other side of the stage, killing itself. But no professional was able to consistently beat the bot. Of 10 professionals that faced the bot, each one was killed more than they could kill the bot. (All the pros played as Captain Falcon against the AI, but most of them mainly played as that character anyway.)
Super Smash Bros Melee might not be entirely solved, the way Go or chess could be categorized, but the MIT and NYU team has shown that even seemingly-complex multiplayer games aren’t safe from being beaten by AI in short order. And as the games—merely testing grounds for AI that will eventually live in the real world—get more complex, so does the ability for a future bot to understand the physical world it inhabits.
But that shouldn’t make us worry about Melee-happy robots—it seems for now that they’re still susceptible to cowering in fear.