We’ve created a bot which beats the world’s top professionals at 1v1 matches of Dota 2 under standard tournament rules. The bot learned the game from scratch by self-play, and does not use imitation learning or tree search. This is a step towards building AI systems which accomplish well-defined goals in messy, complicated situations involving real humans.
I'll admit that I've spent probably more hours than I should have playing Dota, so this really hits home for me. Whereas Chess and Go are turn-based and therefore involve much more discreet choices, Dota is real-time: the entire game is one continuous stream of decisions. Real-time decision-making is obviously a critical step towards OpenAI's goal of AGI.
OpenAI's bot performance is now best in the world in a 1v1 setting, but Dota's competitive scene is mostly focused around 5v5 teams. The next step is training a group of bots to perform at that level. The results have the potential to be truly fascinating—will bot teams play similarly to human teams? My bet is no.
While OpenAI is working on Dota 2, DeepMind is working on Starcraft 2. It's a good time to be a fan of real-time strategy games.Read more...