CHALLENGE
The World Bank task team leader in Mali was having doubts. It was 2014 and Mali has been experiencing instability and conflict since the military coup of 2012 and the occupation of the northern regions by armed groups. School attendance plummeted, and a World Bank-funded project supplied meals at schools as a way to bring students back.
But reports were trickling in that some children weren’t being fed. The team leader needed to know if the reports were true. And he needed to know why. Site visits from a monitoring team were not turning up much. Those are telegraphed well in advance. A large-scale survey would be too expensive and take far too long.
Those issues, moreover, are not specific to any one country or any one project. Supervision missions can provide information, but the evidence they gather is limited to the small number of often carefully chosen projects they visit. Large-scale surveys are data-intensive, time-consuming and expensive. They often deliver results with such delay that their findings are no longer valid.
Another way was needed.
APPROACH
Johannes Hoogeveen, lead economist in the Poverty and Equity Global Practice, came up with a common-sense solution. He would harness the power of cell phone technology and the know-how of locals to get specific questions answered from hard-to-reach project sites. By cutting down the number of questions asked to a bare minimum, focusing on a select set of issues, sending out surveyors alone or in pairs, and collecting data quickly, he was able to get to ground truth.
It took short visits to just 20 schools in and around the capital city of Bamako over two weeks to get answers. Some of the money promised by the government had never been sent to the schools. The money that had been sent was wrong because the formula the government was using had errors. And the money that did get there was delayed.
Armed with that information, the team leader was able to reach out to the government of Mali. Counts of students were improved. Payment delays were cut from three to one month. The number of annual disbursements tripled.
The method Hoogeveen employed in Mali, which he dubbed Iterative Beneficiary Monitoring (IBM), is not complex. Indeed, the whole idea behind it is to strip away the layers of systems that can add cost and intricacy to the business of evaluations without adding to a real base of knowledge of what is happening on the ground. For a fraction of the cost of traditional surveys and site visits, IBM sends local people, armed with several dozen questions on computer tablets, out to hard-to-reach project sites. Questioners go alone on public transport instead of in large groups in expensive four-wheel-drive vehicles. Whenever possible sending enumerators is avoided and interviews are held over the phone. The low cost of data collection makes it possible to go check back every few months and assess whether improvements are sustainable.
“Getting ground truth about these projects is important because without it, we can never fix the wrongs that inevitably happen, the problems that arise even in the best of circumstances,” Hoogeveen said. “Nothing is designed perfectly. I firmly believe that if something is implemented the way you thought it through behind your desk, then you’re doing something very wrong. If you want to create change, you probably need much more frequent feedback. You need that calibration until you get a working system.”
