Why a generation of mobile health apps have surfaced and disappeared

For a few months in 2016 community health workers working with children in northern Malawi had a helping hand.
A smartphone app led them through a standard protocol of examinations helping to boost both the accuracy of diagnoses, as well as their confidence. They told researchers they liked the app because it reassured them that their decisions were solid.
But, after eight months, the project was over and the app removed for fear that – with the funding gone – it could soon become out of date.
This is hardly an isolated incident. mHealth makes a tantalising offer: to plug gaps in health provision in impoverished countries using cheap, widely available tools. But across Sub-Saharan Africa, Asia and South America hundreds – perhaps thousands – of mHealth experiments done in the hope of improving healthcare disappear after a year or two.

Methodological woes

A 2016 review cast a harsh light on mHealth. Led by Alain Labrique, a public health professor who heads the Global mHealth Initiative at Johns Hopkins University in the United States, it concluded that – with some exceptions – “the strength of the findings to support the use of mobile interventions over alternate strategies is still limited”.
The review found that many interventions go undocumented; many are designed without academic rigour, making it hard to gauge their success; and often the chosen outcome measures mean they cannot be compared with other studies.
Caroline Perrin, a PhD student in telemedicine at Geneva University in Switzerland, says that instead of measuring outcomes, studies tend to measure how much health professionals like their new apps, and are often too small to have the statistical sensitivity to detect a fall in mortality, for example.
And this, she says, adds to the difficulty of demonstrating that a single digital element woven into a complex health system actually makes a difference.
Perrin’s professor, Antoine Geissbühler, adds that rigorous evidence is important not just to prove effectiveness, but to avoid harm. Even at pilot stage, he says, training sessions that focus on one health activity can eat up time and distort priorities. And governments need to make hard decisions between deploying eHealth and other activities, such as running vaccination campaigns. “Some countries cannot do [it] all.”
Few studies ever get scaled-up because they cannot prove to donors that they are effective, says Labrique’s review. And many projects are simply unscalable due to poor choices during the design phase.
Choice of software, for example, is crucial, says Garrett Mehl, a specialist in digital health at the World Health Organisation. A bespoke bit of standalone software might be fine for a few villages. But could it withstand the demands of 1,00,000 health workers – the help desks, the training and re-training, the “community of knowledge” that’s in the background, keeping it ticking over?
“The reality is that unless they are using software that’s a ‘global good’ it’s unlikely to be able to scale,” he says. But even then, other factors may come into play.

Take the app that proved useful in diagnosing children in northern Malawi – the Supporting LIFE app (Low-Cost Intervention for Disease Control). “Even though the software was open source, the ministry of health just doesn’t have any internal capacity to roll out the software or even maintain it,” says project’s coordinator John O’Donoghue, now an eHealth specialist at University College Cork, in Ireland.
“Realistically if the project was to be modified and maintained and implemented we’re probably talking about a multimillion Euro effort per year.”

Chasing success

Digital and health communities are trying to find better ways to capitalise on mHealth’s promise.
For example, Perrin is helping researchers choose useful outcome measures when they set up pilots. Although a study may be too small to spot a drop in maternal or neonatal mortality, it might be able to detect an improvement in another parameter shown to decrease mortality.
She has identified 70 such variables, all of which can be influenced by digital interventions – having four check-ups during a pregnancy, for example, or identifying danger signs that predict neonatal illness. These are set to be adopted by WHO as part of upcoming guidance from its Department of Reproductive Health and Research on digital maternal health interventions.
O’Donoghue says such proxies were essential for Supporting LIFE. “Thankfully,” he says, “no child passed away during our study”, so they measured hospital referral rates for evidence that the digital system improved the accuracy of health workers’ referrals.
Adding to efforts to impart rigour to mHealth pilots, a few years ago the WHO, with the International Telecommunications Union, published advice on how to scale mHealth non-Communicable Disease projects. The agency has also developed a lexicon for both health and digital specialists, whose first version was released earlier this year.
Meanwhile, Principles for Digital Development written by a group of donors and implementers were endorsed by many international organisations in 2015, and a supporting forum launched last year.
And the WHO produced the mHealth Assessment and Planning for Scale or MAPS toolkit in 2015, which has been adopted by the new Malawi eHealth Research Centre at Mzuzu University, which O’Donoghue helped found.
This means that anyone proposing an mHealth intervention must go to the Centre and spend several days assessing it against the MAPS toolkit. If it fails to demonstrate it can meet the demands in areas such as human resources, finance and technical capacity, and no solution can be found, the proposal is abandoned.
Toolkits and better study design may improve pilots and perhaps allow some scale-up. But if mHealth is to make a real difference in the developing world it requires scale-up to national level.
Many people fail to grasp how formidable that challenge is, says Mehl. “It requires the enabling environment of policies, the infrastructure, the tools – and knowing which work and which don’t – and the training.”
As a rare example of success, he highlights the Indian government’s messaging service Kilkari, designed by BBC Media Action, which delivers free audio messages about pregnancy, childbirth and childcare to women from the second trimester of pregnancy and continues until the child is one year old. Developed and tested in the state of Bihar in 2013, it went on to reach two million subscribers within 12 months of its launch. The government went on to adopt it in 2016, footing the bill for call costs and rolling it out in 13 states.

Sarah Chamberlain, director of digital for BBC Media Action India, highlights three factors in its success. First, the government had already invested in registering pregnancies and births, and it saw Kilkari as a way of incentivising women to supply this data; second, long-term and “patient” investment by donors; and third, spending years cultivating a relationship with the Ministry of Health. “You cannot imagine how much energy it takes to nurture a relationship with a national government,” she says. “You can’t just fly in and fly out.”

Cultural change

Mehl also believes that moving from local to national scale requires cultural change and a leap of trust: ministers must be convinced about the value of leaving paper behind to enter the digital machine. And this is linked with the long-term funding it takes for training, help desks, hardware and software refreshments.
He says Tanzania’s Ministry of Health is “exceptional” in having undergone just such a journey. Working with the international health organisation PATH, in the Better Immunisation Data or BID Initiative, it is using mHealth to improve the tracking of immunisations in a way that is compatible with other digital health interventions.
The system has scaled to four regions of Tanzania with plans to reach ten more regions soon. Henry Mwanyika, regional director for Africa and digital health at PATH, attributes its success to consultation up and down the health hierarchy – and to “breaking the language barrier” so that engineers, politicians and health workers could communicate with each other.
What most impressed Mehl was the early recognition by the ministry that the project would take at least five years of “burn-in” — meaning “understanding and absorbing and tweaking and optimising”.
This may be a sign that countries are waking up to the complexities of mHealth. This is reflected in a resolution that was adopted by member states at the World Health Assembly this May, and is widely seen as a milestone for catalysing progress in the field, says Mehl.

The resolution seeks a global strategy, technical assistance for scale-up, and information on best practice and developments in digital health.

mHealth may be emerging from its Wild West phase. Indeed, the term itself is being abandoned in favour of “digital health”, to convey the joined-up thinking required for success.
But we might need to subdue our hopes. “The expectation that it’s going to have massive impacts early in the phase of adoption is unrealistic,” Mehl says. “It’s largely helping people to do what they are already doing but a little bit better, faster and more effectively.
“It’s going to be a slow transformation.”

This article first appeared on SciDev.Net.