In the following paragraphs, findings are developed for water supply, sanitation and hygiene, and WASH in schools programmes, by ‘chronological’ order in the programme cycle (whenever the information exists): situation analysis; targeting; programme design and implementation; M&E; effect on the reduction of inequities.
Water supply (39 reports from 28 countries)
Focus on poor or vulnerable areas and people is rarely emphasised in theory of change and logical framework (logframe) for rural water supply programmes. The extent to which water supply interventions are adequately targeting the neediest populations is not well documented in evaluations. Only five water supply interventions have been reported to effectively target deprived and vulnerable areas such as water scarce, flood-prone, cholera or Guinea worm-endemic areas (Ghana 2009 and 2012, Malawi 2011, Sudan 2012, Kenya 2013, Bosnia-Herzegovina 2013). Reaching remote areas is particularly an issue for water supply interventions as there are few skilled local contractors and larger companies are not willing to bid for challenging assignments.
Water supply interventions reportedly face some problems with siting of water points, with community leaders sometimes securing easy access at the expense of poorer and more vulnerable people.
Based on the very limited information available on inclusiveness of water points, it seems that there is a lack of considerations for aged and disabled people. Only one evaluation out of 39 reports good performance in this matter (Zimbabwe 2011). However, this is not an issue reported by interviewed beneficiaries, key informants or evaluators. One can therefore guess that elderly people and those with disabilities rely on others to fetch water.
In terms of people affected or at risk of climate change, only one report mentions rain water harvesting schemes were used as a means to increase water security in areas where there are availability or quality issues with groundwater or surface sources, notably in areas affected by drought and pollution.
No report highlights the fact that the intervention actually generated an extra cost for the beneficiaries, especially the poor in rural areas, because they were encouraged to switch from a free, unimproved water source to an improved but paid service. It also correlates with the fact that in many countries UNICEF is reportedly providing the water point without any contribution to the capital cost from the beneficiary community, and without clear principles regarding water tariff – even when government policies provide such guidelines. This increases access for the poorest, who are usually not asked by the community to contribute to the cost of repairs in either situation. Where a water tariff is applied, UNICEF typically leaves the analysis and decision about the most appropriate tariff to the beneficiary community or, in some cases, to the local authorities responsible for water supply. Based on the evaluation reports, no ‘willingness to pay’ surveys have been commissioned by UNICEF in any of the WASH programmes. This is usually favourable to the poorest because communities tend to be sensitive to equity issues. However, training committees on how to design the appropriate tariff with cross subsidies for the poorest users and how to collect the tariff is also uncommon. It poses a problem in terms of sustainability (see section 6).
In (peri-)urban settings, a couple of reports highlight the fact that the poor have greatly benefitted from the water supply intervention when they previously relied on more expensive water supplies such as water trucking.
In urban areas and middle-income countries there are possibilities to put in place more sophisticated pro-poor financial instruments, tariff or management models but it is not always done – or not always successful. Specific examples are documented in detail in the Egypt 2010, Bosnia Herzegovina 2013, and Somalia 2012 reports (see case studies in section ).
Most evaluations looked at the presence of women in water management committees, which is a requisite of many WASH programmes. It was found that they typically include women as extension workers, secretary or treasurer but rarely at top level decision-making positions. One programme also promoted the participation of the poor (SHEWA-B in Bangladesh, 2014). Their influence in the decision-making remains limited: involvement does not result in actual management of water by women (or the poor). Some of the main reasons are fear of voicing their opinion, belief that men (and community leaders) must decide in development projects, and lack of time.
Some reports underline the lack of a mechanism to ensure bringing up the voice of the poor, women, remote households and other groups in the village water committee and local authorities. Most UNICEF WASH programmes have not engaged in developing consultative platforms and accountability mechanisms involving vulnerable groups. Only two reports describe a good practice in this respect: Bosnia Herzegovina 2013 and Bangladesh 2014.
Some reports state that access to safe water has significantly reduced poverty by improving the health status of the population and reducing water fetching time, which in turn increases working time on the farm or for other activities (Ethiopia 2010, Bolivia 2011, Palestine 2012, Burkina Faso 2012, Rwanda 2013, Indonesia 2013, Burundi 2013, Djibouti 2014, Bangladesh 2014, and Timor Leste 2015). However, these reports rely on key informant interviews, anecdotal evidence or households’ self-reporting only. None of them have attempted to measure this anti-poverty effect using a methodology that allows for attribution to the water supply intervention.
Similarly, some reports suggest that improved access to water in the community increased girls’ school attendance because they spend less time fetching water when at home (Burkina Faso 2009, Bolivia 2011, Ethiopia 2012, and Mali programme evaluation 2015). However, none of them has tried to quantify this effect on a rigorous basis. The overall lack of robust evidence is also a conclusion of broader literature reviews that included non-UNICEF evaluation reports.16
Sanitation and hygiene (48 reports from 31 countries)
In terms of geographical targeting, CLTS programmes typically focus on small rural communities that have higher rate of open defecation. There is a strong association between (extreme) economic poverty and open defecation, so it can be concluded that the poorest people tend to be the ones who benefit the most. This emphasis on poor or vulnerable populations is reflected in most CLTS programme theories of change and logframes.
Some evaluations underline that when CLTS activities are carried out in the field by the district or local sanitation departments, reaching the less accessible communities have proved difficult because of logistical constraints and the limited availability of means of transportation and operating budget.
Women and other categories of vulnerable inhabitants are not especially targeted as CLTS is a community-led approach that covers the whole population without distinction, and the cessation of open defecation must be universal in the village. However, women are particularly interested because sanitation and hygiene related issues are often their responsibility within households and they face many specific challenges related to open defecation. Their need for privacy forces them to defecate very early in the morning, or to hold off until the evening, causing health complications, or walk long distances – especially in areas where desertification is progressing like some parts of India, Madagascar and Sahelian countries. An evaluation conducted in Madagascar in 2014 mentioned that in some communities women used to go in the open with umbrellas to hide behind when they defecate. Going at night exposes them to a range of risks including being bitten by insects or attacked by animals. Gender-based violence is also consistently documented: being harassed and physically attacked by men in the bush or at night, accused of illicit relations and subject to violence or even honour killing (Sudan 2012, Mozambique 2013, Pakistan 2013 and 2014, Madagascar 2014, global CATS evaluation 2014, Mali 2015). Many reports highlight the necessity of being accompanied even at night. Effect of CLTS interventions on women is rigorously evidenced in the impact evaluation carried out in Mali in 2012-2014. The randomised control trial established that women in CLTS villages were significantly more likely to feel that they had privacy when defecating and to feel safe defecating at night than women in control villages. Therefore CLTS succesfully addresses women’s needs specifically. Even though CLTS is community-based, evaluations are finding that special attention is being placed on related gender issues.
Good participation of all villagers is consistently reported, including elderly, children, poor, disabled persons, and of course women (sometimes more than men). This positive finding typically applies to both the triggering session and the subsequent community meetings organised by the community or natural leaders.
Women’s participation in CLTS facilitation teams, sanitation committees or as natural leaders is widespread. It is reported that their participation results in more dynamic and sustainable community engagement around open defecation free (ODF) and sanitation issues. A couple of reports even include concrete examples of women’s mobilisation and empowerment having led to other community initiatives and results in areas other than sanitation (Mauritania 2010, global CATS evaluation 2014). For example, in Bondaguda village in the Odisha state in India, the CLTS-related mobilisation process led to the reorganisation and revitalisation of women’s groups and helped solved water problems in the community. “Situated at the top of a hillock, water scarcity had always been a problem for the village and it was the women who had the arduous chore of fetching water. Now, every house in the village has a rooftop rainwater harvesting structure. In addition, there was a place in the village where, until recently, women were not permitted to climb and sit on the dais. However, as a result of the women’s groups leading the sanitation work, they have been able to break this tradition. They can now organize their own meetings and address the outsiders sitting on the dais. Moreover, the women’s groups have successfully banned the consumption of liquor in the village” (global CATS evaluation 2014).
A limited number of reports highlight the difficulty in maintaining the community-wide approach in strict Muslim context where men and women cannot attend the same gathering. UNICEF programmes do not always properly address these issues. Local facilitators manage this issue with mixed success.
Hygiene messages and facilitation methods are adapted to the capacity of poor and less educated people, as they are mainly based on observation exercises and common sense rather than more elaborate explanations on disease contamination and vectors of transmission. Most programmes promote ash as a more affordable alternative to soap for low-income villages and households.
The poor are said not to have time or enough educational background to volunteer or be nominated as such natural leaders or members of sanitation committees.
Some anecdotal evidence of strong intentional shaming, threats of sanctions or actual penalties against those who do not build latrines have been found in only one report (WCARO 2010), referring to UNICEF and non-UNICEF programmes implemented in Asia and Africa before 2011. In some cases, these punitive measures were decided by the community itself, or came from the own initiative of some CLTS facilitators without UNICEF being involved. The extent to which these measures, out of line with the CLTS spirit of self-help and dignity and within a right-based approach, were applied to poor and vulnerable household is uncertain. There is also no indication that these practices persist in CLTS programmes supported by UNICEF, as emphasised in the global evaluation of ’Community approaches to total sanitation’ (CATS) in 2014.
Self-building of latrines with locally available material and skills is a common principle of most CLTS programmes, in line with the original CLTS approach promoted by Kamal Kar and Robert Chambers. This is pro-poor, as household do not have to spend money – only a few days (an average of six days to build a latrine in Mali 2014). Only a minority of households spend money and these are mostly better-off ones that can afford to build higher standard latrines. In this case inequalities may persist – but not inequities.
Latrines are expected to be adapted to the specific needs of all individual households including female, elderly or disabled people since they are the one responsible for designing and building their own latrine using creativity and available material; no standard model is promoted. However, for latrines to be really inclusive and gender-sensitive, it requires that these household members are consulted during the whole process. Yet, their actual involvement in the decision-making is not clear from the evaluation reports. Some evaluators recommend that UNICEF develop and propose various options and designs that are sensitive to the needs of various categories of population. UNICEF does it already in some countries. This external intervention in the households’ decision-making intends to address the inequity dynamics at household level. However this approach, if not facilitated with strong communication skills, brings the risk of reverting to a top-down approach where the intervention is not entirely led by the community. Guiding households’ decision-making might compromise the philosophy behind CLTS. External evaluators and UNICEF WASH programmes do not seem to be always aware of this risk.
UNICEF leaves it to the communities to decide whether or not and how to assist vulnerable households. It is hoped that with the overall goal and decision to achieve an open defecation free area, people will help each other. As summarised in the Nigeria 2014 report, “if a community wants to become ODF, everyone needs to have access to a latrine and the WASH committee makes sure this happens.” Indeed, solidarity is observed by most evaluators with elderly people, disabled people with family support, households headed by women etc. receiving help from others for digging the pit, gathering the necessary materials etc. In a few countries or communities, however, this solidarity does not happen (DRC 2011) or is paid for (Zambia 2011). The ODF certification of communities suggests that these disadvantaged households effectively gained a latrine – the certification process assumed to be trustworthy, which is doubted by several evaluators. An additional concern is the case of remote households in areas where housing is scattered. It is unclear whether they are considered in the ODF verification and certification process.
In any case, poorest, vulnerable, marginalised and remote households are consistently documented to face more problems in building, maintaining or upgrading their latrines. This is typically substantiated by beneficiary self-reporting. Challenges are especially prevalent in areas where appropriate materials are not readily available locally, or where hydrogeological or climate conditions renders simple structures inadequate and require less affordable solutions: rocky or sandy soil, high water table, areas prone to floods or other extreme seasonal weather. One evaluation did not validate this finding when triangulating with field visits: “By far the most commonly expressed opinion regarding the reasons for reversion to open defecation was that it was related to affordability; the poorest households could not afford to maintain their latrines. However, what the consultant was able to observe did not corroborate this entirely; many poorly maintained latrines belong to families with much better constructed and maintained homes and other outbuildings” (Timor Leste 2015).
A few UNICEF programmes included or still include targeted subsidies to the poorest (Nepal 2013, Pakistan 2014) or reward for latrines built (India 2008 and 2013, Ghana 2012, Mozambique 2014). In some other countries evaluators have recommended introducing them for equity purposes, in order to compensate for difficulties faced by the most disadvantaged groups and for the lack of effective solidarity mechanism within communities. This however goes against the original CLTS approach that strictly forbids household subsidies, and the findings of other evaluations and past rural sanitation programmes shows that such subsidies are expensive (and therefore difficult to take to scale), difficult to manage (targeting issues, potential frictions within communities, coordination with other donors and non-governmental organisations (NGOs) etc.), reinforce dependency mind-set, and do not ensure actual construction, behaviour change and sustainable use of the latrine. In Mozambique, they have been found “influencing communities to move towards immediate quantitative results rather than adopting more durable, embedded behaviour change” (Mozambique 2014). In none of the programmes or evaluations does the choice of subsidies as a pro-poor measure seem to have been the result of a well thought pro/contra analysis of various possible options such as the use of community work, savings and loan mechanisms, microcredit, revolving funds etc. It rather seems to be a relic/legacy/survival of the past approach to rural sanitation based on household subsidies.
To tackle this issue, Sierra Leone (2013) introduced Village Savings and Loan Associations (VSLAs) as a way to help secure some level of access to finance at community level. VSLAs are also seen as a potential way to empower the role for women, as they very often manage the schemes. One report (Pakistan 2014) mentioned a voucher system piloted in the sanitation marketing component of the rural sanitation programme, with vouchers delivered to most vulnerable households to get supplies from the supported marts and entrepreneurs. In Nepal and in India, village-managed funds have been established to assist poor families; microcredit options for the poor have also been proposed by some programmes (Nepal 2013, India 2013, global CATS evaluation 2014).
Other, non-financial options could be explored. In some countries where they are already common, evaluators recommended that UNICEF promote public or shared latrine options to address both the needs and the capacities of the poorest households. However public and shared latrines are reported as dirtier and poorly maintained.
Other WASH programmes confronted with environmental obstacles and affordability issues chose to train local masons in building different types of low cost latrines adapted to the local context.
Despite the inevitable difficulties faced by the households that have less abilities, capacities and resources, it is consistently proven that they are the ones that benefit the most from CLTS interventions, since they are the ones that practice open defecation the most.
The elderly are sometimes reported to slip back to open defecation because open defecation habits are well anchored – while they should be the ones benefitting the most from a nearby latrine.
Disabled people benefit a lot since they do not have to travel long distance, according to several reports.
While increased privacy, dignity and security are the most systematically cited benefits of CLTS interventions for women specifically, two evaluations (Kenya 2013, Pakistan 2014) suggest that at the same time the intervention has added to the workload of women and girls, as they are responsible for cleaning latrines and fetching additional water required due to the introduction of flush latrines or handwashing stations, for example.
The risk of social exclusion of villagers reluctant to engage has not been substantiated by any evaluation evidence.
Regarding hygiene promotion programmes, which have not been much evaluated, reports demonstrate that UNICEF hygiene promotion programmes tend to benefit the literate, upper caste and higher socio-economic groups the most. They are exposed to more communication channels, and understand and retain more hygiene messages (Guinea Conakry and Guinea Bissau 2009, Nepal 2009, Kenya 2012).
It is also shown that women are more targeted by and exposed to hygiene promotion messages. They also retain and benefit more than men from these campaigns, for the reasons already explained above.
WASH in schools (and health centres)(28 reports from 23 countries)
WASH in schools programmes have been evaluated in different countries throughout the period of the UNICEF global WASH strategy, whereas only three evaluations looked into WASH interventions in health centres (Malawi 2009, Ethiopia 2010 and 2012). These three evaluations give much less attention to this component than to other community and school-based components of the WASH programme evaluated. Therefore the below paragraphs primarily focus on school WASH.
The meta-analysis observed that geographical targeting based on equity criteria such as deprived areas and poor access to WASH is an element that is rarely examined in evaluations of WASH in schools. When it is examined, findings are very mixed. Sound criteria have been found to be used in Burkina Faso 2009 report but not in the 2012 report, in Kenya in the 2013 report but not in the 2012 report, in DRC in the 2008 report but not in the 2011 report (criteria not consistently endorsed), only partially in Ethiopia (2012), Burundi (2013) and Indonesia (2014), and not in Mali (2015). This may well be the result of both a lack of equity-focus programming and of budget. Building up-to-standard water points and a sufficient number of latrines for the whole school, and ensuring quality, security and inclusiveness is indeed costly. In some cases UNICEF has chosen to balance intervention schools that were to some extent already WASH-equipped with other requiring more investment. Moreover, easy accessibility has been one of the criteria for selecting beneficiary schools in some programmes, leaving remote ones (often without resource) at a disadvantage.
The meta-analysis also found that WASH inequalities among schools are rarely emphasised in school WASH programmes’ theory of change and logframe, when these were included in the evaluation report. However, the specific needs of girls are mentioned in most programme documents and evaluations – more rarely the needs of children and teachers living with disabilities.
The evidence of school WASH facilities built by UNICEF being adapted to the needs of disabled people is mixed. Approximately a third of the evaluations found that they were not adequately designed, another third found that they were correctly designed but construction departed from original plans, and the remainder that facilities were inclusive overall. Issues commonly observed are: narrow door and cubicle; step at the end of the ramp or leading to the handwashing station; immediate surroundings of the latrine not accessible with a wheelchair etc. Evaluation findings suggest firstly that it is not easy to address all types of disability in the latrine design. Secondly that inclusive WASH facilities alone do not solve the problem of access of children with disabilities to school as in many cases the classrooms, school ground etc. are not disabled access friendly; accessible school infrastructure requires a good coordination between WASH and Education sectors. And thirdly that in many countries children with disability go to specifically adapted schools. In many cases where these schools do not exist, children simply do not go to school, for many other reasons that are not related to WASH. As a matter of fact, several evaluations noted that WASH in schools programmes have installed disabled-friendly latrines in schools while disabled children are enrolled in none of the intervention schools.
Girl-friendly latrines with gender-separation, appropriate siting for girls, elevated walls to ensure privacy, adequate features for menstruating girls etc. have been found in most but not all school WASH programmes’ design (approximately two thirds). A need for revision of latrine designs has been pointed out in Ethiopia 2012, Palestine 2012, Haiti 2013, Liberia 2013, Indonesia 2014, Pakistan 2014 and Timor Leste 2015. In programmes where latrines are gender-separate in theory/design, the practice in the field sometimes differs. Sometimes the latrines are separated in practice but the distance between both blocks is insufficient to ensure privacy (case of Burkina Faso 2012 or Mali 2015 among others). This may be caused by a lack of gender awareness by builders, school management committees and teachers, a lack of consultation with users by builders, or a lack of space in school yard. Even when latrines are geographically separated, utilisation rules set by UNICEF or by the schools are commonly not consistently enforced by teachers or overlooked by younger children themselves.
Many large school WASH programmes have integrated training on menstrual hygiene management and specific features in the hardware package delivered to beneficiary schools (Nepal 2013, Nigeria 2014, Bangladesh 2014, Pakistan 2014, Mali 2015). It cannot be concluded that this is mainstreamed in all UNICEF-supported programme because evaluations failed to consistently look at integration of menstrual hygiene management in the UNICEF programming, certainly because this issue is relatively new on the WASH agenda and because WASH itself has a limited role in supporting girls to manage menstrual hygiene. However, where this dimension is taken into account, it was not always based on a comprehensive assessment of the actual needs of teenage girls. Some of these evaluations noted a lack of access to water in girls’ latrines and of a way to dispose of pads or similar products.
A ratio of girls and boys per latrine is recommended by UNICEF and the World Health Organisation (WHO) when planning the number of school latrines to be built. This is beneficial to girls since they tend to have fewer latrines reserved for them as historically more boys go to school than girls and education ministries have not built new latrines for girls to keep pace with the progressive increase in girls’ enrolment. However, evaluations shows that the application of this ratio is not yet mainstreamed in all UNICEF-supported school WASH programmes – even when a standard ratio has been anchored in national policies or guidelines. The examples of Ethiopia 2012 and Mali 2015 suggest that the programme budget and space available in the schoolyard might be insufficient to systematically apply this recommended ratio. However, its non-application, specifically in large schools, may also be explained by the fact that such a ratio would result in a high number of latrines, requiring too much efforts and resource invested in cleaning (assuming that all latrines would be used).
The participation of girls in health/hygiene clubs is ensured as a principle in most school WASH interventions, and confirmed in all WASH in schools evaluations. There is discussion on whether to set a boys/girls ratio in the membership such clubs – which is the case in several UNICEF programmes – or if selection of members should rather be based for example on motivation, dynamism, and charism. In several reports – not all of them – it is reported that girls’ participation offered them the ability to influence WASH related decisions. The extent to which they actually use this opportunity to take initiatives favourable to them is not documented, however.
Some programmes communicate to schools the message that latrine cleaning should not be a task left to girls only.
Some evaluations observed that women were also specifically targeted and trained as members of pupils’ mothers associations or food sellers. One evaluation documented that women benefitted from the introduction of community saving groups (with exclusive woman membership) as part of the school WASH programme (Mali 2015). One purpose of these groups is to help the community provide the school with the necessary WASH supplies in a sustainable manner, after initial external donation. Concrete examples are given showing that women were effectively empowered and took several WASH and non-WASH related initiatives as a result.
Girls benefit the most from school WASH interventions such as a water points and separate latrines, as they usually are the ones responsible for bringing water to the school which sometimes requires them to walk long distances (they are relieved from this task), tend to be discouraged of using the latrines when they are not gender-segregated, and to be absent during menstruation when there is no appropriate latrine ensuring privacy or no latrine at all.
Key informants, parents, teachers and children consistently state that school WASH improves girls’ enrolment and education. Nevertheless, these qualitative data are not confirmed by robust quantitative analysis. Indeed, the evidence base on the effect of WASH in schools programming on children’s – and especially girls’ – enrolment, absenteeism, cognition and drop out is weak. The only rigorous UNICEF impact evaluation of a WASH in schools programme was conducted in Mali between 2011 and 2014. It are not able to identify any attributable impact on girls’ enrolment. This evaluation report includes a literature review of randomised and matched control trials that also concludes that there is a lack of robust and consistent evidence of significant impact of WASH in schools programming on girls’ enrolment and absenteeism. Most of these impact evaluations have found no effect or were inconclusive. The same conclusion is supported by other, broader school WASH literature reviews. 17
Pupils from rich and poor families benefit equally from school WASH interventions. However, it is well known that enrolment and attendance of children from poorer families is lower. There is no evidence to suggest that school WASH interventions change this story. Evaluations have not attempted to assess the benefits of school WASH interventions on children from poor households specifically, in terms of enrolment, absenteeism, health, learning or success.
Performance related to equity by type of WASH intervention is summarized as follows:
