Report (third draft for comments) Table of content

9.1.On equity

Country	Type of intervention	Population category	Equity aspect	Details / docs
Bosnia-Herzegovina	Water supply	The poor; women; other vulnerable groups	SitAn, M&E, tariffs, participation in decision-making	Bosni-Herz. 2013 (p.33, 35, 39, 41-46, 56, 65). DevInfo database of vulnerable populations developed and being used by multiple actors in Bosnia. Capacity built on methodologies to identify them and developing tools to reach/support them based on their needs. Action plan development and their implementation monitored. Tariff setting and subvention mechanism. Participation of women in Municipality Commissions / MMBs and of women’s Non-governmental organisations (NGOs) in their structures.
Kenya	Water supply	The poor	Financial mechanism / pooled fund	Kenya review report 2013 (p.28-29). Water service trust fund began its operations in 2005 as a pro-poor water services financing mechanism to mobilise capital investment for WASH services benefitting the rural and urban poor. UNICEF’s participation limited because of a lack of engagement and/or inadequate financial procedures.
Egypt	Water supply	The poor	Financial mechanism / revolving fund	Egypt 2010 and add’l note sent by Ihab. Revolving found for poor rural household water connection in Egypt; criteria for economic and social targeting of the most vulnerable within the community (but hard to manage and verify; inclusion and exclusion bias due to favouritism)
Pakistan	CLTS	The poor	Financial instrument / vouchers	PATS eval 2014 (p.60) and add’l info from Kamran. Vouchers delivered to two categories of poor households identified through participatory methods to help them purchase latrine materials
Sierra-Leone & Mali	CLTS & WinS	Women and the poor	Financial instrument / saving groups	VSLAs in CLTS villages in Sierra Leone (2013) and women-managed saving groups in villages surrounding WinS programme in rural Mali (2015)

9.2.On scalability

Country	Type of intervention	Scalability aspect	Details / docs
Ethiopia, Kenya & DRC	WASH (global)	Financing	SWAp / One WASH in Ethiopia and pool fund in Kenya and DRC
Ethiopia	CLTS	Government leadership, institutional arrangements	How upscaling was achieved through government leadership, implementation through government’s structure at regional and local levels. Quality assurance arrangements.
Mali	WASH in schools	Coverage, institutional arrangements, coordination, beneficiary engagement	Long-term partnership / investment and capacity building with multiple NGOs; establishment of REHA to build capacities and reach scale; standardisation of implementation approaches, tools, partnerships documents, M&E system etc. and continuous learning in order to scale up without compromising on quality.
Sierra Leone	CLTS	Encouraged diffusion	National network of natural leaders

9.3.On sustainability

Country	Type of intervention	Sustainability determinant	Details / docs
Mali	WASH in schools	Technical determinants	Mali 2014 evaluation. Systematic examination and revision of the construction project cycle leading to drastic improvements in the quality and sustainability of WASH facilities. Designs and accompanying technical standards and guidelines revised annually based on the feedback from government partners, implementing NGOs, construction companies and engineering firms.
Zimbabwe	WASH in schools	Financial determinants	Zimbabwe 2011 evaluation. School block grant. Look for update.
Somalia	Water supply	Institutional determinants	Somalia 2012 evaluation. Management model of rural water supply services: Public-private partnership (PPP) arrangements to allow for higher service delivery performance, increased sustainability, and pro-poor cross-subsidies. Lessons learnt from this failure. Look for updates.

10.Annexes

10.1.List of evaluation and sustainability check reports

File	Content
	List of all evaluations of UNICEF WASH programmes from beginning of 2007 to July 2015, hyperlink to the reports, rationale for the inclusion/exclusion of each reports in the meta-analysis, shortlist of reports selected for this meta-analysis and name of the main reviewer. (draft version)

10.2.Detailed analysis of the evidence base

File	Content
	Overview and quantitative description of the evaluation evidence: trends in the number of evaluations, geographic coverage, topical coverage, and quality of evaluations reports. (draft version)

10.3.Matrices of content analysis

File	Content
	Completed matrix of analysis of the evaluation reports’ content for each thematic (equity, scalability, sustainability) and quantitative analysis of findings. (draft version)

1^ Sustainability checks are rapid field surveys commissioned since 2009 by several Eastern and Southern African country offices and funded by the Government of the Netherlands. They provide a snapshot of sustainability in UNICEF WASH interventions.

2^ See for example: Johanna Birckmayer, Carol Weiss, Theory-Based Evaluation In Practice: What Do We Learn? Evaluation Review, Vol. 24 No. 4, (August 2000), pp. 407-431, http://erx.sagepub.com/content/24/4/407.full.pdf+html; Nicoletta Stame, Theory-based Evaluation and Types of Complexity, Evaluation, Vol 10-1, (2004), pp.58–76, http://www.stes-apes.med.ulg.ac.be/Documents_electroniques/eva/eva-gen/ele%20eva-gen%207360.pdf

3^ See for example: Carol Taylor Fitz-Gibbon, Lynn Lyons Morris, Theory-Based Evaluation, Evaluation Practice, Vol. 17, No. 2, (1996), pp. 177-184, http://aje.sagepub.com/content/17/2/177.extract

4^ This report refers to CLTS as an individual approach rather than encompassing it under the umbrella term of “Community Approaches to Total Sanitation (CATS)”. UNICEF HQ commonly uses the term CATS rather CLTS because CATS also includes sanitation promotion approaches other than CLTS such as school-led total sanitation (SLTS), sanitation marketing, etc. that are part of some country WASH programmes. Similarly, this report refers to these other sanitation promotion approaches by their specific name – as they are in the individual evaluation and sustainability check reports reviewed.

5^ Derived from: UNICEF Strategic Plan 2014-2017, Realizing the rights of every child, especially the most disadvantaged (2013); Braveman P., Gruskin, S., Defining equity in health, Journal of Epidemiology Community Health (2003), #57, p.254–258, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1732430/pdf/v057p00254.pdf; WHO, Equity in Health Systems, 2015, http://www.who.int/healthsystems/topics/equity/en/

6^ Derived from: DfID, Upscaling field level pilot research experiences, (2003), http://r4d.dfid.gov.uk/Project/2767/; IICD, Piloting and upscaling: Both crucial stages in social innovation (2010), http://www.iicd.org/IICDCorporateBlog/2010/09/08/piloting-and-upscaling-both-crucial-stages-in-social-innovation; English Collins Dictionary.

7^ Derived from: OECD / DAC, Criteria for Evaluating Development Assistance (2007), http://www.oecd.org/dac/evaluation/49756382.pdf; DfID, Sustainable Livelihoods Guidance Sheets (2000), http://www.eldis.org/vfile/upload/1/document/0901/section2.pdf.

8^ http://www.unicef.org/evaldatabase/

9^ http://www.unicef.org/evaluation/index_GEROS.html

10^ Some reports have been excluded for more than one reasons listed above.

11^ Except sustainability checks which are not rated.

12^ Based on the classic programme cycle stages and on the UNICEF Gender Checklist for WASH Managers (2010): http://www.unicef.org/wash/index_key_points.html

13^ Children are not considered among the vulnerable categories of the population here as it is the mandate of UNICEF to focus on them. Poor children are.

14^ Adapted from UNICEF, Evaluation of the WASH Sector Strategy “Community Approaches to Total Sanitation” (2014)

15^ This categorisation of sustainability determinants has been developed by the author based on the existing literature, notably for water supply: USAID/Hodgkin, The sustainability of donor-assited rural water supply project (1994), http://pdf.usaid.gov/pdf_docs/PNABQ942.pdf; Aguaconsult, Assessing sustainability in rural water supply (2003), http://www.aguaconsult.co.uk/assets/Uploads/Publications/WorldBank-AssessingSustainability-2003.pdf; UNICEF, Sustainability of Rural Water Services: Principles and practice (2011), https://intranet.unicef.org/PD/WASH.nsf/0/1BE4FF37713BBBFE85257981006E30C3/$FILE/RWS_Sustainability_TechnicalBrief.pdf. For sanitation and hygiene: Plan Australia, ODF sustainability study in Ethiopia, Kenya, Uganda, Sierra Leone (2013), http://www.communityledtotalsanitation.org/sites/communityledtotalsanitation.org/files/Plan_International_ODF_Sustainability_Study.pdf; UNICEF-DRC, Healthy village program sustainability study in DRC (2014); University of La Plata/Maria Laura Alzua, PEP Network, UNICEF-Mali, CLTS impact evaluation (2014), http://www.cedlas-er.org/sites/default/files/cer_evaluation_files/mali-clts-impact-evaluation-2014.pdf; WSP, Factors associated with achieving and sustaining open defecation free communities in East Java (2011), https://www.wsp.org/sites/wsp.org/files/publications/WSP-Factors-Achieving-ODF-East-Java.pdf; WSP, Long-term sustainability of improved sanitation in rural Bangladesh (2011), https://www.wsp.org/sites/wsp.org/files/publications/WSP-Sustainability-Sanitation-Bangladesh-Brief.pdf; UNICEF-Mali, Literature review on determinants for latrine construction & use and for handwashing with soap (2012); IDS, CLTS and Sustainability: Taking Stock (2015), http://www.communityledtotalsanitation.org/sites/communityledtotalsanitation.org/files/Frontiers_no4_CLTS_and_Sustainability_taking_stock.pdf; WSP, What Influences Open Defecation and Latrine Ownership in Rural Households?: Findings from a Global Review (2014), http://www.wsp.org/sites/wsp.org/files/publications/WSP-What-Influences-Open-Defecation-Global-Sanitation-Review.pdf; UNICEF, Developing and Monitoring Protocol for the Elimination of Open Defecation in Sub-Saharan Africa (2013), http://www.communityledtotalsanitation.org/sites/communityledtotalsanitation.org/files/Thomas_and_Bevan_Elimination_of_open_defecation_SSA.pdf.

16^ See for instance the 3ie WASH evidence gap map http://gapmaps.3ieimpact.org/evidence-maps/water-sanitation-and-hygiene-evidence-gap-map and the draft report from the literature review conducted by UNICEF in 2015 (internal working document).

17^ See for instance the 3ie WASH evidence gap map http://gapmaps.3ieimpact.org/evidence-maps/water-sanitation-and-hygiene-evidence-gap-map and the literature review conducted as part of the UNCIEF Mali 2015 WASH in schools impact evaluation, whose report is included in this meta-analysis.

18^ Based on UNICEF WASH annual reports 2013 and 2014.

19^ Figures based on UNICEF WASH annual reports 2013 and 2014.

20^ See UNICEF WASH annual report 2014.

21^ And also links to the other dimensions of sustainability i.e. financial and technical.

22^ Toilets Can Work: Short and Medium Run Health Impacts of Addressing Complementarities and Externalities in Water and Sanitation, Esther Duflo,Michael Greenstone, Raymond Guiteras, And Thomas Clasen, p.1, http://economics.mit.edu/files/10743

23^ See for example: UNICEF, More than Soap and Water: Taking Handwashing With Soap to Scale, (2008): http://globalhandwashing.org/wp-content/uploads/2015/03/HWWS-More-Than-Soap-and-Water_Training-Module.pdf; Val Curtis, Keynote presentation, WASH Conference, Brisbane, (2011): https://sanitationupdates.wordpress.com/2011/06/03/brisbane-wash-conference-2011-presentations-on-hygiene-and-sanitation/. The same observation is made in: FHDesigns (for Plan), ODF sustainability study, (2013): http://www.communityledtotalsanitation.org/resource/odf-sustainability-study-plan

24^ Proportion of schools with functional latrines where open defecation was not reported or observed.

| Page