Beyond the theory: How lean impact evaluations are transforming WFP programmes
Three real-world examples from Burundi, Nepal and Ghana demonstrate the power of experimental evidence on the ground.
By , , and
Back in 2021, we published a blog outlining our new approach to generating evidence in rapidly evolving humanitarian settings: “Lean” Impact Evaluations.
The idea was simple. Could we harness the power of rigorous experimental methods, like the A/B testing commonly used by tech companies, to optimize WFP interventions?
Fast forward to today, and the answer is a resounding yes.
By removing the need for a “pure control” group (data collected from people outside the programme population) and instead testing variations of programme design, lean impact evaluations have become an invaluable tool for the World Food Programme. They allow WFP country offices to test, learn and adapt in real-time.
To see how this works in practice, let’s look at three recent lean impact evaluations that have shaped WFP operations on the ground.
1. Burundi: Informing supply chain models with A/B testing
The challenge: In Burundi, WFP provides daily meals to hundreds of thousands of schoolchildren. Traditionally, this relies on a centralized, in-kind procurement model, which can be vulnerable to global supply chain disruptions.
The lean evaluation: To support a shift toward Home-Grown School Feeding, WFP and its partners piloted a new commodity voucher model in 2022–2023. Instead of centralized deliveries, funds were transferred to local authorities to procure food directly from local farmer cooperatives. Using an A/B testing approach, schools were randomly assigned to either continue the centralized model or test the new commodity voucher model.
The impact: The results were striking. Schools using the new model saw a massive jump in meal delivery, providing meals to children an average of 13 days per month, compared to just 7.4 days under the centralized model. By sourcing locally, schools faced fewer supply chain disruptions. Although increasing the number of meal days led to a reduction in school meal quality, ensuring consistent caloric sufficiency is crucial. The massive gain in meal days therefore gave WFP the evidence needed to scale the commodity voucher model. It also paved the way for a full-scale impact evaluation, currently underway, to further optimize nutritional and educational outcomes. (Read the report here)
2. Nepal: The power of timing
The challenge: Extreme weather events are devastatingly common in Nepal. But what happens if we provide cash assistance before a forecasted climate shock — like a flood — hits, rather than waiting for the aftermath?
The lean evaluation: Ahead of the 2022 monsoon season in the Karnali River basin, WFP partnered with the World Bank to evaluate anticipatory action. The study compared households that received cash transfers right at the flood peak against a group receiving standard post-shock assistance some weeks later.
The impact: Timing matters a lot. The evaluation revealed that receiving cash early allowed families to significantly improve food security immediately after the floods. The anticipatory action group was also significantly less likely to resort to negative coping strategies (like reducing meal portions) and recipients had lower anxiety and lower depression scores. The study showed that acting early doesn’t just boost food security; it can also protect mental health. (Read the full report here)
3. Ghana: Maximizing yields with a zero-cost tweak
The challenge: Smallholder farmers in Northern Ghana face soaring fertilizer costs and rising poverty, making the agricultural lean season particularly brutal. In 2023, WFP launched the Farmer Support Activity, providing cash transfers totaling approximately US$315 to help farmers weather the storm.
The lean evaluation: Could the schedule of the payment change how farmers invest it? WFP set up a lean impact evaluation comparing two delivery modalities: a single lump-sum payment versus three equal monthly instalments. The total amount of cash provided was identical for both groups.
The impact: A minor operational tweak led to substantial gains in agricultural productivity. Farmers who received the lump-sum payment were able to make larger upfront investments: they cultivated 5% more land, utilized 18% more household labour, and spent 7.5% more on chemical fertilizers. Ultimately, the lump-sum group saw a 13% to 17% increase in crop sales revenue compared to those receiving monthly payments. However, monthly instalments favoured investments in livestock (for example, households bought more livestock). By simply changing how the money was distributed, WFP supports farmers to make different decisions, with significantly different impacts. (Read the full report here)
Evidence that drives action
These three cases from Burundi, Nepal and Ghana highlight the true value of WFP’s impact evaluation strategy. When designed thoughtfully, lean impact evaluations are highly practical, cost-effective tools that answer the operational “how” and “when” questions.
Acknowledging the limitations and trade-offs
While lean impact evaluations offer operational value, they also come with design considerations worth flagging.
Firstly, sample sizes and statistical power must be carefully recalibrated. Unlike designs with a pure control group, A/B designs test arms may perform more similarly to one another, requiring larger samples to detect meaningful differences.
Secondly, they primarily focus on informing specific time-bound operational choices, instead of focusing on testing academic theory like traditional randomized controlled trials. Their findings may not generalize beyond the context and time-period under analysis, and because of their operational nature they tend to focus on short-term changes rather than long term changes.
Finally, and most importantly, because A/B tests intentionally avoid a “pure control” group, they can only measure relative rather than absolute impact. For instance, while we can confidently say that vouchers outperform centralized delivery in Burundi, early cash outperforms late cash in Nepal and lump-sum payments outperform instalments in Ghana, none of these designs by themselves can tell us the overall effect of the programme: whether local procurement improved farmers’ income, whether anticipatory action payments meaningfully changed long-term resilience, or whether cash transfers boosted farmers’ productivity. Designs aiming to advance the broader academic literature on programme effectiveness should complement A/B test designs.
Lean impact evaluations are purpose-built tools for actionable decision making. They are designed to deliver timely evidence that informs specific operational choices, their real measure of success is whether they drive better programmes and decision making.
Want to learn more about our ongoing impact evaluations? Follow the WFP Evaluation Medium blog online.
The Impact Evaluation of the School Meal Programme in Burundi , Nepal Anticipatory Action Impact evaluation and Ghana Farmer support activity are conducted as part of a partnership between the World Bank and the World Food Programme. The impact evaluation teams includes Atekou A., Heirman J., Jeong D., Khincha R., Kondylis F., La P., Lombardini S., Shastry V., Uckat H., Manea, R. E., Adusumalli, N., Christian, P., Dunsch, F. A., El Toukhi, O., Waidler, J., Balantrapu, T., Batmunkh, O., Kelley, E., Lane, G., Malhotra, K., Moreno, S. and Paulose
