What differences are we making? The main takeaways from the Impact Evaluation Forum

In a pioneering initiative, the WFP’s Office of Evaluation hosted an Impact Evaluation Forum from December 4–7, 2023. The event brought together a diverse range of stakeholders, including governments, donors, NGOs, academia, UN agencies, and WFP staff, to share experiences and showcase evidence of the impact of development and humanitarian assistance on people’s lives.

By Ola ElToukhi

The primary goals of the forum were to bridge the gap in evidence generation, enhance practitioner capacity, and inspire the next phase of impact evaluations supported by WFP and its partners.

Here are the key takeaways from the Impact Evaluation Forum:

1 — In the face of resource scarcity and increasing need, evaluations need clear objectives from the start

When faced with limited resources and the choice between implementing and learning, the importance of clearly defining the objectives of an evaluation becomes paramount. All types of evaluations respond to accountability, especially when they are published. However, some evaluations are designed for strengthening institutional capacity and operations and others are for learning about programme effectiveness. We need to understand how impact evaluations will fill an important knowledge gap, building on the existing global evidence base. For example, unconditional cash transfer programmes are extensively evaluated, and they work. We also know that mosquito nets can save lives and that we need to send girls to school. We need more evidence on how cash can be delivered in a way that supports women’s economic empowerment.

2 — Impact evaluations add value in the humanitarian sector when asking how to better design assistance

In the humanitarian setting, the focus should shift from determining whether programs are good or bad to exploring ways to improve them. Addressing pressing needs in humanitarian emergencies like hunger, conflict, and natural disasters, we don’t question whether vulnerable populations should receive assistance or not. And sometimes, in such contexts, we cannot decide between food or cash as we can only give one modality or another. However, we need a nuanced understanding of intervention timing, targeting strategies, and the choice between different modalities (lumpsum vs. small cash bundling) to ensure we reach the most people in the most cost-effective way possible.

3 — It’s time to consider the costs of having too little evidence

It is often said that impact evaluations are too expensive, especially in humanitarian contexts where money goes toward saving lives. However, continuing to implement programmes that do not achieve expectations is both costlier and unethical. There are many ways to significantly reduce evaluation costs including leveraging existing data, standardizing measurements, and building institutional capacity through partnerships.

4 — Harnessing existing data will help close the gap

With advancements in remote sensing, satellite images, and administrative data, we do not have a shortage of real-time data. We need to share and connect these pipelines of data and create a good database to serve monitoring, rigorous evaluations, and prediction purposes. Also, in humanitarian contexts where we cannot always randomize interventions, having good panel data can still help in producing rigorous evaluations through other quasi-experimental methods.

5 — In the UN, collaboration is a keyway to optimize resources

Impact evaluations offer a unique avenue for UN agencies to unite and make the best use of scarce resources. Through collaboration, information sharing (for example sharing procurement systems for partners), and strategic planning, the effectiveness of evaluations can be significantly enhanced. Exploring joint evaluations promotes collaboration, resource pooling, and the utilization of collective expertise. This approach ultimately contributes to evidence-based decision-making across the UN.

The outcomes of this transformative forum signal a new era of impact evaluations and cooperative endeavors within the humanitarian landscape.

Now, in the wake of the Impact Evaluation Forum, WFP’s impact evaluation unit is excited to move forward on further strengthening a community of practice with others interested in generating impact evaluation evidence in the humanitarian space. Over the next year, we will be organizing events together with partners to showcase emerging results from evaluations, share the tools built along the way, and identify ways to work on a common evidence agenda. Interested in learning more about WFP’s impact evaluation work so far? Watch the video, first showcased at the Forum