Impact evaluation in humanitarian and fragile contexts builds a case for aid
Co-published by the evaluation offices of WFP and UNICEF
In December 2024, several international organizations, governments, private foundations and academic institutions gathered at the United Nations headquarters in New York for a forum on impact evaluation in fragile settings. The mood was upbeat, and despite a unanimous urgency to tackle development issues like conflict and climate change head-on, the outlook for collective action was bright.
As development practitioners navigated the fallout of aid cuts across the sector, one message from the WFP-UNICEF Global Impact Evaluation Forum rings truer than ever: impact evaluation is a moral imperative to design effective programming and deliver higher value for money. And while the humanitarian and development sectors are witnessing unprecedented shifts and challenges, impact evaluation could help prioritize programme actions, fostering future generations’ well-being.
More than six months since the Forum, held from 2-5 December 2024 — WFP and UNICEF release video interviews from participants who shared their reflections about impact evaluation, its comparative advantage, and its goals.
Hear first-hand from experts and leaders in the development and humanitarian impact evaluation community how rigorous impact evaluation could steer dwindling aid towards the most pressing problems, in the most cost-effective ways.
Through partnership, UN agencies, donors, governments, non-governmental and academic institutions remain committed to collaborate on a shared impact evaluation agenda.
1. Why impact evaluation?
Impact evaluation builds a case for effective aid. As scepticism of international aid increases, impact evaluation helps reinforce the sector’s standing and help organizations to make the best possible decisions with finite resources. As Florence Kondylis, research manager from the World Bank says, “we owe it to our clients — the poor who live in conflict-affected regions”.
2. Doing more with less: Maximizing impact with limited resources
Cost-effectiveness analysis is crucial for maximizing the value of impact evaluations. Evaluators should build cost analysis into evaluation designs by default. A key action point of the forum was to embed cost-effectiveness analysis in every impact evaluation to demonstrate the value of programmes and give clear guidance to decision-makers on how to drive efficiency. Håvard Mokleiv Nygård, deputy director general, Norad, says every dollar or krone must be spent as cost-effectively as possible to reach more people more effectively.
3. Building bridges: The power of collaboration
The forum reiterated a need to break siloed ways of working, tackling shared learning challenges together. Joint evaluations demonstrate the value of collaboration by promoting accountability and learning across UN agencies and with other partners. It’s critical for joint efforts along the humanitarian development nexus to continue, says Loic Couasnon, assistant evaluation officer, UNHCR.
4. Impact evaluation and the role of governments
The forum emphasised the role of the UN to support governments in translating policy challenges into evaluation questions and rigorous evidence into policy action. National governments can play a key role in building an evaluation mindset and demanding impact evaluation. Joerg Faust, director of the German Institute for Development Evaluation (DEval) says governments can play a role by improving incentive schemes for more impact evaluations.
5. Boosting inclusivity through impact evaluations
Acknowledging that variable impact evaluation capacity exists across regions, countries and institutions, the forum brought together researchers from universities and research institutions in the Global South to discuss impact evaluation capacity building. As Monica Lambon-Quayefio, senior lecturer at the University of Ghana says, this is critical as it pushes diversity into the conversation, ensuring impact evaluations are not exclusively undertaken by researchers from the Global North.
6. Ethical Impact evaluation in humanitarian and fragile contexts
The forum affirmed the need to run impact evaluations in the most difficult contexts; adapting methods to operate in fast-moving, volatile environments is a major priority for keeping impact evaluation relevant. Amber Peterman, associate research professor at the University of North Carolina, says evidence in these contexts remains scarce, and although there is often a hesitancy to collect data due to ethical issues, we can and we should collect it in a safe way for vulnerable populations.
7. Navigating uncertainty: innovation in fragile settings
Qundeel Khattak, senior research advisor for humanitarian cash and voucher assistance, Save the Children, says it’s important to keep sharing innovative approaches in impact evaluation, especially in fragile settings. Some outcomes from the forum included making wider use of A/B testing, as well as innovative methods such as geospatial analysis and machine learning-based causal inference to test and refine what works in fragile contexts.
8. Unlocking potential: policy-driven impact evaluation
The most vulnerable people worldwide deserve decisions that affect their lives to be based on rigorous evidence. As global needs rise and resources struggle to keep pace, we don’t have the luxury of doing what we think might work — or worse, do what doesn’t work or what might inadvertently be doing harm. Rather, we need to take evidence-based decisions on how to best achieve our end goals. Rigorous evidence matters. When we have it, we must use it; when we don’t, we should produce it.
Riaz Lodhi, country director of WFP El Salvador says the results from impact evaluations ensure that the resources from donors go to the right people at the right time.
