Four lessons from a year of the WFP’s School-Based Programme Impact Evaluation Window
Jonas Heirman (Senior Evaluation Officer, WFP), Roshni Khincha (Research Analyst, World Bank), Florence Kondylis (Research Manager and Lead Economist, World Bank), Simone Lombardini (Evaluation Officer, WFP), Astrid Zwager (Research Officer, World Bank), and colleagues working on the School-Based Programme Impact Evaluation Window.*
In early 2021, the World Food Programme’s Impact Evaluation Unit teamed up with WFP’s School-Based Programme division and the World Bank’s Development Impact Evaluation (DIME) department to launch the School-Based Programmes Impact Evaluation Window. Like other windows, the objective is to contribute to the evidence base for school-based programmes while simultaneously supporting local evidence needs for WFP country offices.
Through creating the knowledge agenda for this window and engaging with many country teams, we learned four key lessons:
1. There are many answered questions
To identify evidence priorities, we conducted an extensive literature review and consulted with academic experts, country offices, regional bureaus, and programmes. The literature tells us that school feeding interventions increase children’s school enrolment, but many areas require further evidence. For example, the optimal balance between cost, size, frequency, and meals’ composition; the most effective bundle of complementary interventions to improve learning, health, and nutrition; effects on girl’s well-being, intra-household allocations, and whether sourcing and preparing food locally has additional benefits for communities.
We identified four understudied areas through consultations:
- The impact of different programme interventions on children’s nutrition, health, education, and learning.
- The impact of programmes on girls’ well-being, specifically.
- The impact of different school feeding procurement systems on local economies.
- Impacts on household consumption and food security in the presence of shocks.
Teams from many WFP country offices participated in impact evaluation training sessions and discussions about the key topics relevant to their country context. Four countries were ultimately selected to kick-off the window: Burundi, Guatemala, Jordan, and The Gambia.
2. Assessing the overall impact of school feeding is challenging because few programmes are new or expanding to new schools.
New school feeding interventions, or existing programmes expanding to new schools, provide the best conditions to assess the impacts of school feeding on the outcomes we’re interested in. However, few programmes are expanding to new schools.
Among the four countries, only The Gambia has a programme anticipated to expand to new schools. Specifically, the Gambia Agriculture and Food Security Project is expanding its sustainable home-grown school feeding (HGSF) Programme, which allows us to study the impacts of HGSF programmes on children’s outcomes and local economies.
3. Therefore, comparing different procurement modalities is vital for programme learning
Many of the school feeding programmes we have engaged with since 2021 are planning to pilot new procurement modalities for their school feeding programmes; shifting from centralized procurement and delivery systems to modalities that seek to engage local communities and workers. The rationale is to develop local markets and positively impact local cooperatives, farmers, and workers, providing the programmes with a double-edged opportunity to impact lives. We will work with these teams to assess impacts on both service delivery (for example, does local procurement allow for more diverse fresh meals that are better suited to local taste?), and the local economy.
- For instance, in Jordan, the WFP Country Office, as part of direct and technical support to the national government, is piloting a shift from centrally procured date bars and high-protein biscuits (using imported ingredients) to using community-based kitchens that hire local workers who procure and distribute pastries, fruits, and vegetables sourced from local farmers and bakeries.
- In Burundi, the country office is piloting a decentralized procurement model based on direct cash transfers to Provincial Education Department (DPE) which will procure commodities from cooperatives to supply schools. The current centralized procurement model (based on direct food distribution to schools) is prone to procurement and delivery delays resulting in not all school feeding days being met. Moreover, the large food deliveries often do not reach remote areas. The pilot will assess whether a decentralized model can overcome these challenges and increase the number of meals provided, while allowing for more local, fresh ingredients.
Even in countries with a history of HGSF modalities, governments are working to improve links between farmers and institutional markets to boost impacts on the local economy.
- For example, in Guatemala, the government requires 50 per cent of school meals to be procured locally, but, in practice, many schools struggle to meet this target due to a combination of factors like transaction costs of searching and dealing with many small suppliers. The CO will pilot the School Feeding Management App, which aims to facilitate linkages between schools and farmers. The app allows schools to put out orders and farmers to make offers, creating a virtual marketplace with more information available. As part of the pilot, we will study whether this leads to more transactions and more transparent prices.
4. Start small and grow over time
For the three countries, these procurement modalities are new. Therefore, a first-order question is the operational feasibility of these models for delivering meals and their quality, quantity, and diversity.
For this reason, some countries in the Window use a phased approach to assessing impacts.
- For example, in Guatemala and Burundi, the new modalities will start at a small scale to build implementation capacity. The pilots or lean impact evaluations will also provide time to build monitoring systems that can provide adaptive evidence to inform the programmes as they mature. If pilots improve service delivery and impacts can be expected further down the Theory of Change, large-scale impact evaluations will be included in the scale-up of the programmes. Another benefit of starting small is that further experimentation can be used to test strategies to overcome unexpected challenges during the pilots.
Moving forward
We are looking for opportunities to support programmes in generating rigorous evidence to enhance their impact. Please get in touch if you are working on school-based programmes and are planning a new programme, expanding an existing one, or changing procurement modality. We would be happy to discuss the feasibility of inclusion into the Window.
*Colleagues working on the School-Based Programme Impact Evaluation Window include Benedetta Lerva, Cox Bogaards, Dahyeon Jeong, Erin Kelley, Gregory Lane, Hannah Irmela Uckat, Minh Phuong La, Paul Christian, Thiago De Gouvea Scot de Arruda, and all the colleagues from Country Offices, Regional Bureaus, and HQ’s School-Based Programmes.
If you would like to receive updates on new blogs, events or impact evaluation publications, please fill in this form to receive regular updates or contact oev.impactevaluation@wfp.org for more information.
A variation of this blog was published on the World Bank’s Let’s Talk Development portal
