From theory to practice in impact evaluation: making it work at WFP
By Jonas L. Heirman and Simone Lombardini
As demand for impact evaluations in the United Nations World Food Programme (WFP) surges, WFP country offices increasingly recognize their value in shaping effective programmes. Yet, navigating the complex process of impact evaluation can be challenging.
To support this, WFP’s Office of Evaluation has introduced the Impact Evaluation Process Guide along with a technical note on quality standards, designed to streamline the impact evaluation journey and enhance programme impact.
As part of the WFP Impact Evaluation Quality Assurance System (IEQAS), these guidelines act as a roadmap to everyone involved in the impact evaluation process, from WFP staff to external partners.
Technical note on quality standards for impact evaluations
The technical note on quality standards for impact evaluations describes how high-quality impact evaluations must meet quality standards in three core areas:
- Technical requirements
- Use of the evidence and
- Process
WFP’s technical requirements for impact evaluation ensure the evidence generated is of high quality from a methodological perspective and can contribute to global knowledge. It presents the definition of the analytical framework; discusses the type of permissible impact identification strategies; provides considerations on the choice of impact measurement tools; guides on sampling and statistical power; and provides some guidance on data analysis.
WFP’s standards for evidence use ensure that each evaluation aims to contribute to both a global scientific audience and a programme and policy audience. It ensures that evidence is framed within already existing global evidence (for example, informed by literature reviews) and contributes through either academic novelty or through replication; guides on the use of impact evaluation pilots; provides design considerations such as multiple treatment arms, heterogeneity analysis, and qualitative data; and advises on principles for reporting.
Finally, the quality standards for the process help to ensure that the evaluation is conducted ethically and in a way that fosters collaboration and learning. It includes guidance on the independence and transparency of evaluation teams; requirements for ethical clearance from Institutional Review Boards (IRBs); adherence to data protection rules and regulations; and explicit considerations for equity and inclusion (e.g. gender, disabilities, minorities).
The technical note on quality standards for impact evaluations offers a clear, high-level overview of WFP’s expectations, complementing the Impact Evaluation Process Guide. Together, they equip WFP teams and partners with the tools to conduct quality impact evaluations in humanitarian settings.
WFP’s impact evaluation process guidance
Impact evaluations are more than just data collection and analysis. They need to be incorporated within ongoing or planned operations, and involve multiple stakeholders, complex designs, and ethical considerations. Attention to how the evidence is generated is crucial to ensure sustained demand and use.
The impact evaluation process guide supports all phases of the impact evaluation process, from planning to dissemination.
It outlines the roles and responsibilities of each stakeholder during an impact evaluation, provides a roadmap to the processes and outputs in each phase (ensuring robust and credible impact evaluations, with transparent and participatory processes), prioritises ethical considerations, and maximises learning.
Impact evaluation in action
The technical note on quality standards and the impact evaluation process guide are not merely a theoretical framework. They are practical tools used by WFP’s Impact Evaluation Unit and programme staff to select and conduct high-quality impact evaluations.
By providing clear guidance and setting high standards, it ensures that WFP’s impact evaluations are relevant, robust, and useful. As the world’s largest humanitarian agency, this is critical in steering a path towards better programmes and policies that are more effective in achieving WFP’s mission of ending hunger and malnutrition.
