artificial intelligence

Why tech giants are investing millions in Artificial Intelligence that can play video games

AI just beat a top human professional in the game Dota 2, but the technology could help with much bigger strategic problems.

Artificial intelligence researchers at Elon Musk’s OpenAI project recently made a big advance by winning a video game. Unlike recent AI victories over top human players in the games of Go and poker, this AI breakthrough involved a game that many people haven’t heard of, Dota 2. But to the hundreds of millions of fans of this type of online multiplayer battle game, a computer that can beat a professional player is a big deal.

It’s also significant to AI researchers, especially those in companies such as Google, Facebook, Microsoft and IBM, which are investing millions of dollars in creating superhuman AI players for digital games. As AI becomes ever more important in our society, it could have wider implications for all of us because of what it demonstrates about computers’ ability to “think” strategically.

What was particularly remarkable about the Dota 2 victory, achieved by a bot created by the billion-dollar non-profit research company OpenAI, was that its developers didn’t program it with deep understanding of game strategies. Instead, they used an approach known as deep reinforcement learning, where the computer starts with only rudimentary knowledge of game strategy.

By playing against itself millions of times, the AI learns to differentiate good move decisions (that lead to victory) from bad ones. The knowledge is stored in a huge data matrix containing millions of numbers, updated after every self-play game. These numbers encode what’s known as a “function”, the instructions that specify the AI’s learned strategy for every possible game situation. So after the AI researchers programmed the method for learning, the machine effectively taught itself how to make good move decisions.


Dota 2 is part of the massively growing eSports movement, where hundreds of millions of players watch their (human) heroes playing video games, online or in large stadium events. The top human players at Dota 2 are really, really good. They are millionaires who practice for ten hours per day, six or seven days per week. They have lucrative sponsorship deals, professional trainers, sports psychologists, strict health and fitness regimes and many of the other things you would associate with professional players in football or tennis.

So as an AI achievement, beating top human professionals Dendi, Sumail and Arteezy ranks up there with beating human world champions in chess, Go and other games. This is especially true since Dota 2 involves a rich selection of tactics that play out on the screen in real time, meaning players have much less time to think than in turn-based board games.

There are some caveats. The OpenAI player won a two-player version of what is usually a ten-player team game. And each player could only play as one particular character in the game out of over typical 100 possibilities. So this is like beating an individual pro basketball player in a one-on-one game, a significant step that still falls short of the goal of beating a team of human professional players.

Shortly after the show match with Dendi, members of the large crowd were challenged to find ways to beat the AI player, with the first 50 being awarded prizes. All 50 prizes were claimed by humans adopting wacky strategies that the AI player had not previously seen, although the AI can now learn and adapt by itself so would avoid making the same mistakes again.

Why invest in game AI research?

The reason all this is of interest to blue-chip companies is that eSports games provide an easy performance measure that generates substantial public interest. Big firms have been investing vast sums in winning games for more than 20 years, since the triumph of IBM’s Deep Blue against the world chess champion, Garry Kasparov.

The real world is not that simple, and nor is reaching the goal of “artificial general intelligence” comparable to that of humans. But AI’s victory in Dota 2, just like in other games before it, could point to other exciting developments.

For one thing, games designers and players don’t want AI that can simply win a game but also make it more fun. Games provide a unique way to understand how people behave and in particular how human psychology interacts with AI behaviour. By capturing the data for millions of players, as we’re doing at the UK’s Digital Creativity Labs, we can effectively run a huge online psychology experiment that informs us as to what people want from AI, as we research new AI techniques.

Developing AI that can learn to make the best decisions in games could also feed into AI for making other strategic choices in the real world. The Dota 2 AI learns the “function” that gives it the strategy to follow any game situation. Similarly, we could imagine AI programs that learn functions for certain economic, environmental and health situations – for example a recession or an outbreak of disease. These functions would generate effective strategies for dealing with these situations, capable of suggesting good decisions in government or business.

One of the limitations of this kind of decision-making AI is that it can’t tell us why it makes a particular move. While AI may be able to help us make better decisions for some of the toughest strategic problems we face, we will still need humans in the decision loop to consider wider ethical and social considerations. Which will make getting humans and AI to work together more important than ever.

Peter Cowling, Director of IGGI and DC Labs, Professor of Computer Science, University of York.

This article first appeared on The Conversation.

We welcome your comments at
Sponsored Content BY 

Relying on the power of habits to solve India’s mammoth sanitation problem

Adopting three simple habits can help maximise the benefits of existing sanitation infrastructure.

India’s sanitation problem is well documented – the country was recently declared as having the highest number of people living without basic sanitation facilities. Sanitation encompasses all conditions relating to public health - especially sewage disposal and access to clean drinking water. Due to associated losses in productivity caused by sickness, increased healthcare costs and increased mortality, India recorded a loss of 5.2% of its GDP to poor sanitation in 2015. As tremendous as the economic losses are, the on-ground, human consequences of poor sanitation are grim - about one in 10 deaths, according to the World Bank.

Poor sanitation contributes to about 10% of the world’s disease burden and is linked to even those diseases that may not present any correlation at first. For example, while lack of nutrition is a direct cause of anaemia, poor sanitation can contribute to the problem by causing intestinal diseases which prevent people from absorbing nutrition from their food. In fact, a study found a correlation between improved sanitation and reduced prevalence of anaemia in 14 Indian states. Diarrhoeal diseases, the most well-known consequence of poor sanitation, are the third largest cause of child mortality in India. They are also linked to undernutrition and stunting in children - 38% of Indian children exhibit stunted growth. Improved sanitation can also help reduce prevalence of neglected tropical diseases (NTDs). Though not a cause of high mortality rate, NTDs impair physical and cognitive development, contribute to mother and child illness and death and affect overall productivity. NTDs caused by parasitic worms - such as hookworms, whipworms etc. - infect millions every year and spread through open defecation. Improving toilet access and access to clean drinking water can significantly boost disease control programmes for diarrhoea, NTDs and other correlated conditions.

Unfortunately, with about 732 million people who have no access to toilets, India currently accounts for more than half of the world population that defecates in the open. India also accounts for the largest rural population living without access to clean water. Only 16% of India’s rural population is currently served by piped water.

However, there is cause for optimism. In the three years of Swachh Bharat Abhiyan, the country’s sanitation coverage has risen from 39% to 65% and eight states and Union Territories have been declared open defecation free. But lasting change cannot be ensured by the proliferation of sanitation infrastructure alone. Ensuring the usage of toilets is as important as building them, more so due to the cultural preference for open defecation in rural India.

According to the World Bank, hygiene promotion is essential to realise the potential of infrastructure investments in sanitation. Behavioural intervention is most successful when it targets few behaviours with the most potential for impact. An area of public health where behavioural training has made an impact is WASH - water, sanitation and hygiene - a key issue of UN Sustainable Development Goal 6. Compliance to WASH practices has the potential to reduce illness and death, poverty and improve overall socio-economic development. The UN has even marked observance days for each - World Water Day for water (22 March), World Toilet Day for sanitation (19 November) and Global Handwashing Day for hygiene (15 October).

At its simplest, the benefits of WASH can be availed through three simple habits that safeguard against disease - washing hands before eating, drinking clean water and using a clean toilet. Handwashing and use of toilets are some of the most important behavioural interventions that keep diarrhoeal diseases from spreading, while clean drinking water is essential to prevent water-borne diseases and adverse health effects of toxic contaminants. In India, Hindustan Unilever Limited launched the Swachh Aadat Swachh Bharat initiative, a WASH behaviour change programme, to complement the Swachh Bharat Abhiyan. Through its on-ground behaviour change model, SASB seeks to promote the three basic WASH habits to create long-lasting personal hygiene compliance among the populations it serves.

This touching film made as a part of SASB’s awareness campaign shows how lack of knowledge of basic hygiene practices means children miss out on developmental milestones due to preventable diseases.


SASB created the Swachhata curriculum, a textbook to encourage adoption of personal hygiene among school going children. It makes use of conceptual learning to teach primary school students about cleanliness, germs and clean habits in an engaging manner. Swachh Basti is an extensive urban outreach programme for sensitising urban slum residents about WASH habits through demos, skits and etc. in partnership with key local stakeholders such as doctors, anganwadi workers and support groups. In Ghatkopar, Mumbai, HUL built the first-of-its-kind Suvidha Centre - an urban water, hygiene and sanitation community centre. It provides toilets, handwashing and shower facilities, safe drinking water and state-of-the-art laundry operations at an affordable cost to about 1,500 residents of the area.

HUL’s factory workers also act as Swachhata Doots, or messengers of change who teach the three habits of WASH in their own villages. This mobile-led rural behaviour change communication model also provides a volunteering opportunity to those who are busy but wish to make a difference. A toolkit especially designed for this purpose helps volunteers approach, explain and teach people in their immediate vicinity - their drivers, cooks, domestic helps etc. - about the three simple habits for better hygiene. This helps cast the net of awareness wider as regular interaction is conducive to habit formation. To learn more about their volunteering programme, click here. To learn more about the Swachh Aadat Swachh Bharat initiative, click here.

This article was produced by the Scroll marketing team on behalf of Hindustan Unilever and not by the Scroll editorial team.