Adapting contexts, evolving approaches: implementation insights from climate & resilience impact evaluations in WFP
By Ola ElToukhi, Roxana Elena Manea, Jennifer Waidler, Jonas L. Heirman and Chloe Monica Fernandez
Climate change is one of the major causes of humanitarian crises and food insecurity globally.
The first round of impact evaluations selected from the World Food Programme’s climate and resilience window, delivered in partnership with the World Bank’s Development Impact (DIME) department, look at the impacts of programmes that aim to support the resilience of vulnerable communities in the Sahel (Mali and Niger), South Sudan, and Rwanda.
This blog reflects on how these impact evaluations supported WFP programmes over the past three years and highlights some lessons learned during implementation.
Impact evaluations must adapt to contexts and programme needs
In September 2019, a four-day technical workshop was held in Dakar to launch the Sahel Climate and Resilience Impact Evaluation Programme. WFP programme teams from the six Sahel countries and the WFP Dakar Regional Bureau, in collaboration with evaluation experts from WFP’s Office of Evaluation and DIME, discussed impact evaluation design opportunities and their implementation within the forthcoming climate and resilience programme. This facilitated mutual understanding, enabling the evaluation team to grasp the programme’s various activities, while also helping the programme team understand the requirements and timelines of an impact evaluation.
However, the onset of COVID-19 in 2020 necessitated adjustments to timelines and designs to accommodate the new context.
Implementation timelines were adjusted in all countries. In Mali, the government distributed an additional COVID-19 safety net to recipients of the integrated resilience package. The impact evaluation design needed to be modified to ensure it captured the additional support provided, without depriving eligible households of vital assistance. This was accomplished by ensuring that the same criteria were used for targeting COVID-19 support in both the treatment and comparison groups (see Figure 1).
Impact evaluations strengthen WFP monitoring systems
Adherence with the impact evaluation design contributes to the success of the evaluation. In the context of food or cash for assets programmes in South Sudan, it was necessary to ensure adherence at the village, down to the smallest unit level, in this case, the household.
At the village level, the impact evaluation team developed a short survey for WFP’s field offices to input information on monthly asset activities and payments progressively, as they happened. An online dashboard was then used to visualize the asset and payment schedules to identify delays or other obstacles faced by the field offices. The survey and dashboard are highly versatile but were only created as a pilot to serve the impact evaluation. Depending on the need, they have the potential to become country-wide tools to monitor interventions across various sources of funding in real-time.
At the household level, adherence is achieved if those selected, actually receive it. The impact evaluation team digitized the work attendance and payment records for the selected locations to enable adherence checks at the household level. Implementing partners had not been systematically sharing this information with their respective WFP field offices, and they only kept paper records of the location-beneficiary information. The impact evaluation was a good opportunity for the programme teams to pilot tools for implementing partners to supply the monitoring information digitally and enable quick verifications and analyses.
Impact evaluations can evolve into other types of evaluation in WFP
Impact evaluations are just one type of evaluation supported by WFP, alongside centralised and decentralised evaluations. This versatility allows WFP to employ a comprehensive range of evaluation methods to address crucial questions, even when conducting an impact evaluation is not feasible.
In the Democratic Republic of Congo (DRC), the impact evaluation team initially planned a randomised impact evaluation of a joint resilience programme in the Kivu region. However, due to deteriorating security conditions, the original design became unworkable. After consultation with implementing partners and the donor, the impact evaluation in the DRC transitioned to a decentralised evaluation employing different approaches of theory-based and qualitative methods.
Additionally, a complementary quantitative study is underway to comprehend critical factors related to: (i) gender equality and women’s empowerment, and (ii) social cohesion in the eastern DRC.
In conclusion, impact evaluations serve as a powerful tool for learning what works in some of the most challenging operational contexts. However, using impact evaluations for learning in WFP necessitates contextually appropriate and adaptable designs, as well as responsive support for impact evaluation efforts.
Also read: Fluctuating food insecurity in the Sahel: how to help households absorb shocks
