Report (third draft for comments) Table of content

1.Introduction

This section describes the background, purpose and objectives of the meta-analysis, as well as the intended audience. The structure of the report is outlined in the last subsection.

1.1.Background

WASH is crucial to ensure children’s survival and development. The human right to water and sanitation was recognised by the United Nations General Assembly on 28 July 2010 through Resolution 64/292.

WASH has long been an important part of UNICEF’s programme and advocacy work. UNICEF’s first WASH strategy was approved by the Executive Board in 1995 and the second strategy more recently in 2006. In 2014, WASH expenses exceeded 725 million USD, making it the third largest area of expenses for the organisation after Health and Education. Going forward, WASH will continue to be an important area of work, through both regular programmes and humanitarian action. UNICEF’s Strategic Plan 2014-2017 assigns key importance to supporting improvements in water supply, access to sanitation and hygiene behaviours, and commits the organisation to achieving measurable outcomes in these areas. Meanwhile, the new set of post-2015 Sustainable Development Goals (SDGs) have been adopted by the Member State of the United Nations in September 2015, and a new UNICEF global strategy for WASH is in preparation. It is therefore timely to look back and reflect on lessons learnt from WASH programme evaluations to inform UNICEF’s current and future work to support better WASH for children around the world.

1.2.Purpose

The primary purpose of this meta-analysis of WASH evaluation reports is to provide a critical review of and to learn from the current global WASH Strategy (2006-2015) in order to inform the development of the new WASH strategy that will guide UNICEF’s action toward the SDGs during the period 2016-2030. It is also expected that the meta-analysis will contribute to the improvement of the quality of UNICEF WASH programming in the field.

Its secondary purpose is to feed into the current, most vivid, discussions around aid, both internally within UNICEF and in the broader WASH sector, by bringing new evidence-based insights.

1.3.Objectives and intended audience

The overall objective of the meta-analysis is to assess the performance of WASH programmes supported by UNICEF in the timeframe of the current global WASH strategy in terms of equity, scalability, and sustainability, based on past evaluations commissioned by UNICEF.

The meta-analysis therefore aims to answer the following question with respect to equity, scalability and sustainability: What was the performance of UNICEF-supported WASH programmes in these areas, what may have affected performance, and what are the lessons learnt?

These three thematic focuses were jointly chosen by the Evaluation Office and the WASH section for three main reasons.

1. 
   They were included as key components in the global WASH strategy and as a result there were expected to guide UNICEF programming in the field between 2006 and 2015.
   
2. 
   They are highly likely to become even more important for UNICEF and all agencies working in the field of WASH as they are at the core of the post-2015 agenda with the Open Working Group proposal for SDG 6 to “ensure availability and sustainable management of water and sanitation for all”.
   
3. 
   A need has been identified for bridging the knowledge gap in these three areas.
   

The meta-analysis therefore intends to provide the UNICEF Programme Division, and more particularly the WASH sections at headquarters, regional and country levels, with useful insights into its investments and programming as well as a storehouse of programming designs, implementation strategies and lessons learnt. These may be of interest to other WASH and non-WASH development agencies outside UNICEF.

The meta-analysis also intends to provide UNICEF’s offices and sections commissioning and managing (WASH) evaluations – including the Evaluation Office in New York HQ, the Monitoring and Evaluation section in regional offices, and the Planning, Monitoring and Evaluation section in country offices – as well as external evaluators with a critical analysis of how well equity, scalability and sustainability are documented and assessed in thematic evaluations commissioned by the organisation.

1.4.Structure of the report

This report is divided into nine main parts. The methodology used for collecting and analysing evaluation evidence is described in section 2, as well as the limitations related to this meta-analysis exercise. Section 3 provides a description of the evidence base: coverage and quality of the evaluations reports. The findings regarding the three thematic focusses are developed in the subsequent sections (equity), 5 (scalability) and 6 (sustainability). Conclusions, lessons learnt and discussion for both UNICEF WASH programmes and evaluations are presented in section Error: Reference source not found. Section proposes a list of prioritised recommendations for the UNICEF’s future global WASH strategy, for future UNICEF-supported WASH programmes and evaluations. Short country case studies are included in section .

2.Methodology

This section presents the methodology used in the meta-analysis: overall framework, key concepts, and steps for gathering and analysing the evidence base. Limitations are acknowledged. Contributors to and timeline of the overall assignment are specified.

2.1.Methodological approach

The current UNICEF global WASH strategy and Strategic Plan do not provide an analytical framework, indicators, performance standards or good practices against which success in the areas of equity, upscaling and sustainability should be measured, and that could be used as guidance or benchmark for the purpose of the meta-analysis. Therefore, a methodological approach was developed for this exercise specifically. It is inspired by the theory-based approach to evaluation, as it not only assesses the performance of the evaluated programmes, but also explores the how and why of WASH programmes’ success or failure. This approach is particularly relevant when the intention is to open the ‘black box’ of programmes, understand the multiple factors influencing performance, and formulate recommendations that address the identified bottlenecks.² This approach was also chosen because it is particularly useful when examining outcome and impact level phenomena such as equity, scalability and sustainability, which were not systematically observable at the time of the evaluative exercise – because they didn’t occur or cannot always be expected to reach a measurable magnitude in the given period of time.³

The methodology used for the meta-analysis consists of four successive phases:

1. 
   Unpacking and understanding what is to be evaluated by developing a detailed definition of the key concepts: WASH; UNICEF-supported WASH programme; evaluation; equity; scalability; sustainability;
   
2. 
   Gathering and selecting the evaluation evidence to be included in the meta-analysis, making sure that it is robust and representative enough;
   
3. 
   Based on the definitions of key concepts and on the evaluation evidence, assessing the overall performance of UNICEF WASH programmes and of the various types of WASH interventions;
   
4. 
   Examining the robustness of the assumptions that were made, and the underlying determinants and conditions that actually supported or inhibited performance, in order to suggest corresponding corrective measures to the appropriate stakeholders.