artificial intelligence

Why tech giants are investing millions in Artificial Intelligence that can play video games

AI just beat a top human professional in the game Dota 2, but the technology could help with much bigger strategic problems.

Artificial intelligence researchers at Elon Musk’s OpenAI project recently made a big advance by winning a video game. Unlike recent AI victories over top human players in the games of Go and poker, this AI breakthrough involved a game that many people haven’t heard of, Dota 2. But to the hundreds of millions of fans of this type of online multiplayer battle game, a computer that can beat a professional player is a big deal.

It’s also significant to AI researchers, especially those in companies such as Google, Facebook, Microsoft and IBM, which are investing millions of dollars in creating superhuman AI players for digital games. As AI becomes ever more important in our society, it could have wider implications for all of us because of what it demonstrates about computers’ ability to “think” strategically.

What was particularly remarkable about the Dota 2 victory, achieved by a bot created by the billion-dollar non-profit research company OpenAI, was that its developers didn’t program it with deep understanding of game strategies. Instead, they used an approach known as deep reinforcement learning, where the computer starts with only rudimentary knowledge of game strategy.

By playing against itself millions of times, the AI learns to differentiate good move decisions (that lead to victory) from bad ones. The knowledge is stored in a huge data matrix containing millions of numbers, updated after every self-play game. These numbers encode what’s known as a “function”, the instructions that specify the AI’s learned strategy for every possible game situation. So after the AI researchers programmed the method for learning, the machine effectively taught itself how to make good move decisions.


Dota 2 is part of the massively growing eSports movement, where hundreds of millions of players watch their (human) heroes playing video games, online or in large stadium events. The top human players at Dota 2 are really, really good. They are millionaires who practice for ten hours per day, six or seven days per week. They have lucrative sponsorship deals, professional trainers, sports psychologists, strict health and fitness regimes and many of the other things you would associate with professional players in football or tennis.

So as an AI achievement, beating top human professionals Dendi, Sumail and Arteezy ranks up there with beating human world champions in chess, Go and other games. This is especially true since Dota 2 involves a rich selection of tactics that play out on the screen in real time, meaning players have much less time to think than in turn-based board games.

There are some caveats. The OpenAI player won a two-player version of what is usually a ten-player team game. And each player could only play as one particular character in the game out of over typical 100 possibilities. So this is like beating an individual pro basketball player in a one-on-one game, a significant step that still falls short of the goal of beating a team of human professional players.

Shortly after the show match with Dendi, members of the large crowd were challenged to find ways to beat the AI player, with the first 50 being awarded prizes. All 50 prizes were claimed by humans adopting wacky strategies that the AI player had not previously seen, although the AI can now learn and adapt by itself so would avoid making the same mistakes again.

Why invest in game AI research?

The reason all this is of interest to blue-chip companies is that eSports games provide an easy performance measure that generates substantial public interest. Big firms have been investing vast sums in winning games for more than 20 years, since the triumph of IBM’s Deep Blue against the world chess champion, Garry Kasparov.

The real world is not that simple, and nor is reaching the goal of “artificial general intelligence” comparable to that of humans. But AI’s victory in Dota 2, just like in other games before it, could point to other exciting developments.

For one thing, games designers and players don’t want AI that can simply win a game but also make it more fun. Games provide a unique way to understand how people behave and in particular how human psychology interacts with AI behaviour. By capturing the data for millions of players, as we’re doing at the UK’s Digital Creativity Labs, we can effectively run a huge online psychology experiment that informs us as to what people want from AI, as we research new AI techniques.

Developing AI that can learn to make the best decisions in games could also feed into AI for making other strategic choices in the real world. The Dota 2 AI learns the “function” that gives it the strategy to follow any game situation. Similarly, we could imagine AI programs that learn functions for certain economic, environmental and health situations – for example a recession or an outbreak of disease. These functions would generate effective strategies for dealing with these situations, capable of suggesting good decisions in government or business.

One of the limitations of this kind of decision-making AI is that it can’t tell us why it makes a particular move. While AI may be able to help us make better decisions for some of the toughest strategic problems we face, we will still need humans in the decision loop to consider wider ethical and social considerations. Which will make getting humans and AI to work together more important than ever.

Peter Cowling, Director of IGGI and DC Labs, Professor of Computer Science, University of York.

This article first appeared on The Conversation.

Support our journalism by subscribing to Scroll+ here. We welcome your comments at
Sponsored Content BY 

Do you really need to use that plastic straw?

The hazards of single-use plastic items, and what to use instead.

In June 2018, a distressed whale in Thailand made headlines around the world. After an autopsy it’s cause of death was determined to be more than 80 plastic bags it had ingested. The pictures caused great concern and brought into focus the urgency of the fight against single-use plastic. This term refers to use-and-throw plastic products that are designed for one-time use, such as takeaway spoons and forks, polythene bags styrofoam cups etc. In its report on single-use plastics, the United Nations Environment Programme (UNEP) has described how single-use plastics have a far-reaching impact in the environment.

Dense quantity of plastic litter means sights such as the distressed whale in Thailand aren’t uncommon. Plastic products have been found in the airways and stomachs of hundreds of marine and land species. Plastic bags, especially, confuse turtles who mistake them for jellyfish - their food. They can even exacerbate health crises, such as a malarial outbreak, by clogging sewers and creating ideal conditions for vector-borne diseases to thrive. In 1988, poor drainage made worse by plastic clogging contributed to the devastating Bangladesh floods in which two-thirds of the country was submerged.

Plastic litter can, moreover, cause physiological harm. Burning plastic waste for cooking fuel and in open air pits releases harmful gases in the air, contributing to poor air quality especially in poorer countries where these practices are common. But plastic needn’t even be burned to cause physiological harm. The toxic chemical additives in the manufacturing process of plastics remain in animal tissue, which is then consumed by humans. These highly toxic and carcinogenic substances (benzene, styrene etc.) can cause damage to nervous systems, lungs and reproductive organs.

The European Commission recently released a list of top 10 single-use plastic items that it plans to ban in the near future. These items are ubiquitous as trash across the world’s beaches, even the pristine, seemingly untouched ones. Some of them, such as styrofoam cups, take up to a 1,000 years to photodegrade (the breakdown of substances by exposure to UV and infrared rays from sunlight), disintegrating into microplastics, another health hazard.

More than 60 countries have introduced levies and bans to discourage the use of single-use plastics. Morocco and Rwanda have emerged as inspiring success stories of such policies. Rwanda, in fact, is now among the cleanest countries on Earth. In India, Maharashtra became the 18th state to effect a ban on disposable plastic items in March 2018. Now India plans to replicate the decision on a national level, aiming to eliminate single-use plastics entirely by 2022. While government efforts are important to encourage industries to redesign their production methods, individuals too can take steps to minimise their consumption, and littering, of single-use plastics. Most of these actions are low on effort, but can cause a significant reduction in plastic waste in the environment, if the return of Olive Ridley turtles to a Mumbai beach are anything to go by.

To know more about the single-use plastics problem, visit Planet or Plastic portal, National Geographic’s multi-year effort to raise awareness about the global plastic trash crisis. From microplastics in cosmetics to haunting art on plastic pollution, Planet or Plastic is a comprehensive resource on the problem. You can take the pledge to reduce your use of single-use plastics, here.

This article was produced by the Scroll marketing team on behalf of National Geographic, and not by the Scroll editorial team.