Report (third draft for comments) Table of content

7.Conclusions and discussion

The main conclusions and lessons learnt regarding equity, scalability and sustainability are summarised and discussed in this section. A distinction is made between the ones related to the performance of UNICEF WASH programmes and others related to the evaluation of such programmes.

7.1.On WASH programmes

Equity

Summary of performance

Situation analysis

Geographical targeting

Programme design & implementation

M&E system

Results in actually reducing inequities

The poor and most deprived

Women and girls

Aged and disabled people

The remote and hard to reach

Other vulnerable or marginalised groups

Water supply

CLTS

WASH in schools

Conclusion

The meta-analysis found that the performance of WASH programming in terms of equity is mixed.

The effect of UNICEF-supported WASH programmes on the lives of women and of the poorest populations is relatively well documented in the evaluation reports and positive especially for women and girls, mainly based on qualitative evidence. The benefits for the other categories of vulnerable population (remote and marginalised groups, religious and ethnic minorities, aged and disabled etc.) is more limited or unknown.

When a positive effect is reported, it is to a large extent an ‘automatic’ or ‘mechanic’ effect. It is due to the fact that deprived and vulnerable populations are typically the ones who suffer the most from the lack of improved access to WASH, and beneficiary communities tend to apply internal solidarity mechanisms in favour of those who need it most. However, UNICEF’s added-value in supporting the most vulnerable people is too modest, especially where internal equity-related awareness mechanisms do not exist. Based on the evidence collected, UNICEF’s contribution to supporting equity through WASH programming seems to be often restricted to the promotion of women’s participation in local management structures (water point management committees, village sanitation committees, and school health clubs) and sex-separated latrines in schools, health centres and public places. There is still room for improvements in terms of specific support provided to the other population categories.

Geographical targeting of vulnerable and marginalised groups is in many cases inadequate, especially for water supply and WASH in schools interventions. In-depth equity analysis aiming to inform the design and implementation of UNICEF WASH programmes are rare. Involvement of the various categories of beneficiaries to ensure at an early stage that their needs and priorities are taken into account in programme design is also uncommon. Rural water supply and sanitation programmes do not make sufficient use of pro-poor financial and non-financial instruments. Equity-lensed monitoring and evaluation systems are not in place. These shortcomings make it challenging to measure whether UNICEF WASH programmes are actually contributing to the reduction of inequities. Yet, evidence from 12 evaluations in Sub Saharan Africa, Asia and Latin America shows that WASH programmes do not fundamentally empower women, the poorest and the most vulnerable because they fall short in sustainably influencing dynamics of inequities in the local decision-making process and distribution of benefits related to WASH. In addition, the evidence (including UNICEF evaluations that refer to the broader academic literature) for the impact of WASH in schools on girls’ education is not yet robust and consistent.

Globally, CLTS interventions have been found the most equity-sensitive of all types of WASH interventions. They typically focus on small rural communities with higher rates of open defecation and poverty. They encourage a large participation from all villagers through the community-based approach and adopt facilitation methods that are well adapted to the less educated among them. They promote the use of local construction materials and techniques that require no cash disbursement. They benefit everyone (the cessation of open defecation must be universal in the village), especially women as well as the aged and disabled people.

Overall there is still room for improvement for UNICEF to integrate the equity agenda consistently and adequately in its WASH programming, especially in water supply and WASH in schools. More reflection is needed to provide WASH ‘leaving no one behind’.

Discussion (to be completed)

Scalability

Summary of performance

Spontaneous diffusion

Encourages or organised replication

Institutional uptake

Effective and scalable approach

Unit costs

Funding

Main-streaming

Government willingness and leadership

Institutional arrangements, partnerships and coordination

Absorp-tion capacity

Implemen-tation guidelines and tools

Demand and supply sides

Water supply

CLTS

WASH in schools

Conclusion

The performance of UNICEF WASH programming in terms of scalability and upscaling is overall moderate, except for CLTS where it is higher.

The meta-analysis suggests that the main constraint for water supply and WASH in schools and health centres interventions is not the unavailability of financial resources but rather the lack of effective and scalable approaches and the lack of absorption capacity in-country. Funds are made available by governments or external support agencies when both of these preconditions are met, as demonstrated by CLTS The CLTS approach, which is being scaled up successfully by UNICEF, has benefitted from an operational strategy that is easy to communicate, from low implementation costs, and from intense advocacy and mainstreaming efforts from UNICEF.

Spontaneous diffusion of WASH interventions and results is not well documented. Evidence shows however that such a diffusion is limited, mainly because (hardware) WASH interventions are usually technology and cost intensive. Improvements in sanitation and hygiene related behaviours and capacities (software) require intensive and repeated efforts that are not easily internalised by beneficiaries and stakeholders without external support. While encouraged or organised replication beyond the programme initial intended boundaries can strongly contribute to the efficiency and impact of WASH programmes, it is neither formally included from the outset as an objective in the design of most programmes, nor integrated in programme logframes and intervention strategies described in evaluation reports. It could be better planned for and driven by UNICEF but also by governments and donors. Evaluations in some countries found that UNICEF played an important role in bringing together stakeholders who previously worked in silos in development settings, to share experiences and tools developed in regular programming and to encourage their replication. Cases like these illustrate the strength of UNICEF being both an actor in development and emergency settings and taking on the role of WASH cluster coordinator in case of an emergency.

UNICEF’s efforts have been intense and successful in mainstreaming CLTS ‘from the top’. This low cost approach is the only type of WASH intervention so far that has demonstrated both effectiveness and scalability. In the many and rapidly increasing number of countries where it is applied, it has been able to overcome the two typical bottlenecks: availability of funding and absorption capacity in country. The scale of UNICEF’s water supply interventions and, to a lesser extent, of WASH interventions in schools, is still significant. However, the likelihood of these types of intervention to be taken to a larger scale will probably depends on UNICEF’s ability to draw the lessons from the CLTS experience and develop innovative approaches comparable to CLTS.

In terms of advocacy and mainstreaming, one area of reflection that seems to have been neglected so far is incentive mechanisms. Evaluations have identified only a limited number of cases where UNICEF has helped governments put in place organisational reforms, sound administrative procedures and routine activities aiming to create incentives for all stakeholders at national and local levels. National benchmarking systems, competition with reward, or individual administrative evaluations are tools that could motivate all ministerial departments, local authorities, development agencies, service providers and communities to prioritise WASH, search for good practices, perform better and scale up what works. Great achievements may be possible with limited or no external financial support.

There is little evaluation evidence of successful urban WASH programming. Evaluations reviewed outline the challenges of implementing infrastructure-based sanitation interventions at scale in urban and peri-urban settings because of higher unit costs and land tenure issues. The effectiveness and scalability of CLTS in these areas are also questioned notably because social capital, buy-in and ownership tend to be weaker than in rural communities. No example of success has been reported in the areas of (peri-)urban water supply delivery, management models, or tariff setting.

In conclusion, a number of options still need to be explored by UNICEF WASH in order to scale up results to significantly contribute to the achievement of universal access, especially for water supply interventions and WASH in schools.

Discussion (to be completed)

Sustainability

Summary of performance

Technical

Social

Financial

Political and institutional

Quality of programme design and implementation

Contextual and environmental factors

Water supply

CLTS

Hygiene

WASH in schools

Conclusion

The performance of UNICEF WASH programming in terms of sustainability is moderate, except for the promotion of hygiene behaviours in general, and of handwashing and drinking water treatment in particular, where it is weaker.

The sustainability of drinking water provision in communities and schools, most often measured by the functionality rate of the water point at any given time (regardless of the age of the infrastructure), ranges between 60% and 90% depending on the type of technology, the geographical area, the season etc. It slowly decreases over time. These figures are consistent with the literature and with the performance of other development agencies. However, the longer term sustainability of water quality and the level of capital replacement is not well known. The main factors explaining cases where the water point is found dysfunctional are the following: the planning phase did not take into account the population size, leading to overuse of water points; the field supervision during the construction phase was loose, causing low quality infrastructure and technical defects; and the community management (including cost-recovery mechanisms) and the service monitoring and regulation system in place were weak, explaining the lack of preventive maintenance and sometimes delays in repair.

The level of sustainability of latrine use one to five years after ODF certification is comparable to the one of water supply, with a progressive decrease over time. The widespread concern around the sustainability of CLTS does not seem substantiated by the quantitative evidence available, unless the practice of handwashing is taken into account in this measure as a criteria for ODF status. Based on the evidence reviewed, actual practice of handwashing is the CLTS component that is driving its level of sustainability down. Moreover, the meta-analysis suggests that technical and financial determinants as an explanation for reversion to open defecation are probably overestimated by the sector. On the contrary, WASH programmes may not pay enough attention to institutional determinants and the enabling environment. Additional critical barriers include the lack of willingness of households to upgrade their latrines, and the time and efforts needed to support lasting changes in behaviour through frequent follow up visits by implementing partners, technical back-up support and incentives. Hence, mitigation strategies consisting of providing financial support to the poorest households for improving the quality of their latrines and strengthening the supply side might not be an appropriate or sufficient answer to the challenge of ODF sustainability. The hygiene challenge, the lack of attention given to the institutional environment – administrative monitoring and incentives, continuous training and supervision, etc. – and the short timescale of implementation are equally found in school WASH.

The effective and sustainable adoption of handwashing is the greatest challenge for UNICEF, with no more than 5% to 25% of people estimated to wash their hands with soap or ash even shortly after intensive promotion, according quantitative data from 20 reports. It is a concern given its demonstrated low cost and high health impact. It is also a cross-cutting issue for all WASH interventions in communities, schools as well as health centres.

While there are many determinants of sustainability in WASH, the two most problematic ones identified in this meta-analysis are first the institutional factors: lack of governments’ leadership and capacity, weak enabling environment, insufficient regulation, monitoring and incentive mechanisms, turnover of staff etc. They are difficult to influence and would take time to address. One area of reflection and potential improvement is the integration of WASH within existing administrative structures and procedures, and the institutionalisation of incentive mechanisms. The second major challenge is the quality of programme design and implementation, including the geographical concentration of WASH interventions and of integration with other sectors. This can be addressed more easily and quickly, and would have immediate consequences for the sustainability of UNICEF WASH interventions.

In conclusion, as sustainability is now given priority both in the WASH sector and globally, concrete actions could be taken to place it more at the core of UNICEF global WASH strategy and field programming, not only in water supply and sanitation but also and foremost in hygiene promotion.

Discussion (to be completed)

Overall conclusion

The highest performing WASH intervention in terms of equity, scalability and sustainability is CLTS. UNICEF interventions in water supply and WASH in schools and health centres are less performant especially in terms of scaling up. Regarding these two types of interventions, UNICEF needs to continue its reflection and learn from its successful experience with CLTS. The key dilemma to be resolved is: Given the new global ambition set in the SDGs, the limited absorption of national governments and the fragile environment in which it operates, how can UNICEF achieve results at scale without compromising on quality of programming and sustainability?