A number of evaluations and sustainability checks were conducted at least one or two years after construction (or rehabilitation) of the water points. They were in the position to measure actual sustainability based on a few indicators, with more or less robust and quantitative data collection methods and analysis. The other reports provide more qualitative data based on interviews with key informants and field surveys. They are useful in identifying the causes for good and bad performance and discussing the way UNICEF WASH programming addressed them.
The first and most common indicator used to measure actual sustainability of water supply systems installed by UNICEF is the functionality rate. Unfortunately, there is no harmonised definition and collection method used to inform this indicator. Some evaluations and sustainability checks conduct a field survey in a more or less representative sample of intervention areas and count the proportion of water points that are functional at the time of the survey. Others rely on interviews with users and ask whether the water point has been dysfunctional in the past, with a varying recall period. The definition of ‘functional’ also varies: For example, how long must the water point be down to be considered as dysfunctional? Can a water point without adequate drainage be considered functional? Should water quality be taken into account in the definition of the indicator? Furthermore, data on the same WASH programme in the same country sometimes significantly vary from one year to another, as illustrated in sustainability check reports. This may be due to small sample sizes. It is therefore difficult to compare available data and build an overall picture. However, when considering observation-based surveys only, the functionality rate ranges on average from 60 to 90% at any given time, with some more extreme values in a few cases (35% in Zimbabwe 2011, 94% in Malawi 2012). This range is consistent with what is found in the literature. The Mozambique 2013 sustainability check indicates a progressive decrease in the functionality of water points over time, which is also consistent with the literature (10 percentage point lost every three to six years following construction). Reports reviewed also confirm that boreholes and rainwater harvesting systems tend to be more fragile than standpipes, private taps, wells and sources fed by gravity. The mechanical spare parts of the handpump and the pipes used in rainwater harvesting systems are regularly found to be broken, while more simple and robust systems are less prone to damage.
Time to address a water point breakdown is also an indicator that is used in many reports to measure actual sustainability of the service. Due to the lack of data from routine technical monitoring, the indicator is typically informed by users’ self-report which are prone to bias. In general, the time needed for repair ranges between a few days to several weeks, and in some cases months (for example in Burundi 2013). During this period, people revert to other sources of drinking water including unsafe sources. Another indicator is the percentage of water points that are able to provide water all year round. Approximately 80% of the reports using this indicator report that the water is not continuously available especially at the end of the dry season. Again, this issue affects the sustainability of the service and provoke public health issues. The water quality is very rarely measured in the reports reviewed, because of the lack of administrative monitoring data and of the lack of time and budget to carry out on-site sampling and testing. Similarly, evaluation and sustainability check reports provide no data on the level of capital replacement, which would be a good sustainability indicator for longer term sustainability.
When digging into each one of the sustainability determinants and attempting to assess how UNICEF performed in addressing each of them, one notes a remarkable consistency across all evaluation and sustainability check reports. Regarding underlying issues, UNICEF’s approaches and report’ conclusions are similar in almost all countries and over time. The major findings presented below can be considered as quasi universal and only cases of exception are cited.
Technical determinants can explain some of the failures, dry water points and low water quality that translate into low sustainability rates. The choice of building hand dug or shallow wells in low water table or arid areas prone to drought reveals a lack of proper hydrogeological studies and poor technology choice. As a result, the water point may provide insufficient yield or completely dry out before the end of the dry season. Similarly, the lack of preliminary tests often explains cases were the water quality is not complying with WHO standards only a few months after the system was installed. It is obvious from the reports reviewed that technical preliminary studies are not yet systematic in all UNICEF water supply interventions despite the availability of considerable knowledge and guidance in the global WASH sector. When poor sustainability is due to low quality of construction, it is sometimes specifically a problem of incomplete technical standards and design used by UNICEF. Several evaluations point out the lack of attention given to details in standards and design. For example, absence of baffles and drinking trough around the borehole can cause damage by cattle. Absence of specifications on building material and reinforcement techniques causes cracking in walls, parapets and drainage pipes. Absence of specifications on length/height measures, slopes, manholes and soakaways can prevent proper management of water pressure and proper drainage of wastewater. Lack of detailed standards for minimum duration of well development or minimum acceptable yield can led to dry water points. Lack of (enforced) standard for the maximum distance between the water point and the users may lead to the water point being abandoned while the lack of observance of the maximum number of users per water point – usually defined in the national water policy – leads to the water point being overused and quickly damaged. These are the most frequently cited technical flaws in the pre-construction phase. They are partly or entirely attributable to weaknesses in the conception and planning of the water supply intervention by UNICEF.
Issues in the construction phase around the distribution of responsibilities between stakeholders, procurement and contract management, beneficiaries’ involvement and quality control arrangements that also lead to poor quality construction and low sustainability rates are discussed above and in the WASH in schools subsection below. In the post-construction phase, with reference to the technical maintenance of the water system, the weaknesses quoted in most reports are the lack of routine and tools for preventive maintenance, the difficulty to find skilled mechanics locally, and the inaccessibility of spare parts. They partly explain the high rate of breakdown and long downtime of some infrastructure. Concerning the supply chains for repair materials and spare parts, which is a sector wide issue that cannot be tackled by UNICEF alone, it seems that UNICEF has invested more efforts in advocacy, system strengthening and coordination in ESA countries than in other regions. Good practices and progress have been documented in South Sudan (2009), Ethiopia (2010 and 2012), Rwanda (2011), Mozambique (2012, 2013 and 2014), Zambia (2013) and Kenya (2013). In Zambia, spare parts shops have been refurbished and restocked, and have established price lists. In the North of South Sudan, UNICEF promoted the involvement of the state and the development of a locally managed revolving fund for bulk purchase of spare parts. In Ethiopia, UNICEF moved away from offshore procured supplies and engaged instead with private suppliers where available and with the government in the other areas and helped put in place a similar revolving fund mechanism. In Mozambique, successive sustainability checks found that UNICEF intervention communities gained increased knowledge on where to get spare parts and their costs; community leaders connected with individual mechanics or artisans’ associations and 64% even signed a written agreement with them. The same practice was observed in Kenya between water operators and local artisans that had been previously trained and recorded in a database with support of UNICEF. The problem of the small number of local mechanics, the limited training offered by the government and UNICEF, and the geographical mobility of the mechanics that have been trained is noted in several reports (Burkina Faso 2009 and 2012, Somalia 2012, Kenya 2013 etc.).
Monitoring of groundwater resources and quality has been given little attention by UNICEF although the lack of knowledge and monitoring of the resource will increasingly affect the sustainability of water supply delivery. The only documented exception is the Darfur region in Sudan where such a system was put in place during the emergency crisis with strong support of UNICEF. The main bottleneck is the lack of government resources (vehicles, reagents, equipment, and skilled staff) and prioritisation to develop an appropriate system in the risk prone areas and run it on a routine basis.
Besides the technical determinants described above, financial determinants also contribute to influencing the sustainability of water supply. In particular, the lack of cost recovery mechanisms at service level explains why some dysfunctional water points are not repaired or rehabilitated, or why it takes time to gather the amount of money needed to carry out the repair. In all countries, the primary contributors are theoretically the users themselves, through the local management committee or operator in charge of creating an operation and maintenance (O&M) fund and collecting fees. The fund can be financed by an upfront contribution from the community members when the infrastructure is built and/or, more often, by a tariff regularly paid by the users. Compliance of water tariffs with national policies has not been confirmed in any of the reports reviewed. Moreover, in the absence of socio-economic studies carried out prior to the installation of a new water system, the tariff is based neither on the local users’ willingness and ability to pay nor on a viable business plan. It often results in tariff being too high for certain categories of users and total revenues too low for covering all recurrent costs that should be paid for to ensure maximal sustainability. With regard to tariff setting, one original example of cross-subsidy was identified in the Burkina Faso 2009 evaluation report, in which the tariff depends on observable socio-economic characteristics of households: single headed households, elderly people, livestock etc. The majority of reports mention that the water fees are not paid on a periodic basis, but rather ad-hoc whenever there is an actual need for repair – so the O&M fund exists only virtually. When it effectively exists, it is often poorly managed. Consequently, water committee members have to carry out their duties on a voluntary basis, engendering loss of motivation and high turnover rates, and major repairs become unaffordable. In response to this well-known and quasi universal situation, it is the duty of UNICEF and government partners to ensure that committee members are equipped with more skills on how to collect more funds from the community through public campaigning, raising funds from other sources, maintaining financial records and balancing the books. The few evaluators that have looked into the content of committee members’ and local authorities’ training conclude that these aspects are not addressed (Indonesia 2009, Egypt 2010, Rwanda 2011, Liberia 2012, Indonesia 2013, Bosnia Herzegovina 2013).
In this context, it is often admitted or per policy that the cost of major repairs will be borne by the local authority. Financial issues arise at this level too, however, given the lack of budgetary resource and/or prioritisation. As a result, in practice the cost ends up being borne by the governments and external support agencies including UNICEF when donor funds and technical assistance should rather be used to help create a sustainable mechanism for financing water operations and maintenance country-wide. A few reports document or propose interesting ideas in this regard, to be further explored by UNICEF: pre-paid water meters; regional loan facilities for water management committees or private operators; and performance-based financial arrangements for water operators.
Social factors influencing sustainability of water supply and UNICEF’s performance in addressing them are less documented in the reports than other sustainability determinants. The dimension that is most frequently looked at is community ownership. The Ethiopia 2012 report states that “from the focus group discussions and key informant interviews we observe that community contributions during construction enhanced community ownership and participation in follow up and monitoring during implementation. [However] about 67% of respondents still report that government is responsible for the maintenance of the schemes when they break”, and not the community itself as per policy. This quote summarises the conclusion of many other reports with some exceptions (notably Ghana 2012 and Timor Leste 2015). The lack of local ownership and the dependency syndrome revealed by the quantitative survey in the Sudan 2012 report is also documented specifically for water supply in the Burundi 2013 report, which estimates that “many returnees had been accustomed while living abroad in refugee camps to get their basic needs met in the form of delivered aid. It is a drastic change for them to face the expectation that they must now take charge and be responsible for their own future with minimal external support (…). In addition, training and accompaniment of communities and local government during the project period have had a too limited scope to bring about clear understanding and means required for a shift to a local responsibility and leadership”.
The systematic involvement of women in water management – but not in actual decision-making – is discussed in the equity section above (section 4.4). The involvement of community leaders and pre-existing local committees is also mentioned in a couple of reports as key for supporting ownership and increasing collection of water fees. Yet, evaluation evidence is too limited to draw conclusions on how well UNICEF WASH adequately took this element into account in its programming. The same applies with reference to the importance of ownership, social capital, cultural beliefs, behavioural norms and community rules. These aspects are largely overlooked by evaluators – which may suggest a similar lack of consideration in WASH programme design and implementation. The two exceptions worth mentioning are summarised in the following extracts: “There is a social attitude that water is a free good, rather than an economic good” because of the religious belief that water is provided by God and therefore should not be paid for (Egypt 2010). “At the community level, the evaluation team heard that some consumers were reluctant to pay water tariffs, which were as little as 100-150 Philippines pesos per month. In fact during discussions with community members, the evaluation team observed that most of them were paying up to 1000 pesos for their mobile telephones but were reluctant to pay 100 pesos for safe drinking water!” (Philippines 2013). Lastly, the virtuous circle of user satisfaction and water service sustainability is mentioned in only two reports (Egypt 2010 and Djibouti 2014).
Institutional factors that are specific to water supply services mainly relate to management models for service delivery and governance, monitoring, regulation and accountability mechanisms at all levels. They often influence the technical and financial performance of the service (breakdown rate, time to fix the problem, capacity to undertake major repair) and thus its sustainability.
In terms of management models, the most common one is community-based and formalised in the form of a water management committee, also called water point committee, users’ association, WASHCO or WASHCOM. It is composed of representatives designated or elected by the community members. It theoretically has well defined attributions and a time-limited mandate. Theoretically, it is placed under the supervision of the local authority or of a public agency. The main advantage expected from such a model is community ownership. The main drawback is the limited competence of committee members. To overcome this challenge, UNICEF provides training but reportedly to only some members: 2-3 members on average are found to be trained. The same weaknesses are identified here as in other UNICEF capacity building initiatives: the topics addressed in trainings are not based on needs assessments and therefore not comprehensive enough. They tend to neglect local O&M modalities, basic accounting, fees collection, hygiene promotion, and community mobilisation methods. Post-training reference documents and visual materials are often not provided. Turnover and lack of institutional memory are mitigated through refresher training in some programmes, but several evaluations call for institutionalised training centres and routine. Regional ‘Water and Sanitation hubs’ have been established in the Philippines with such a mandate but they were unfortunately funded by donors only, and provided services for free. The importance of official recognition of the community-based water committees and the formation of associations is highlighted in several reports, notably for subsequently obtaining financial support from local government (Ethiopia 2012, Indonesia 2013, Nigeria 2014, Djibouti 2014, Timor Leste 2015 and others). Signs of UNICEF’s engagement in this field are not available in any report however, although it has been known for many years as a success factor and promoted as a good practice by other organisations.
Alternative management models exist, involving private operators in particular. They are well documented in the literature. Little evidence of UNICEF engagement in this direction has been found, although several evaluators judged these models likely to be more sustainable (Rwanda 2011, Somalia 2012, Kenya and Philippines 2013). It is expected that these private service providers, ideally recruited through a competitive process, have more technical and financial expertise and that their time-bound contract creates an incentive for performance and therefore sustainability. The experience of UNICEF in this area is mentioned but not assessed in two reports (Kenya and Philippines 2013). The Somalia pilot mentioned above was the focus of an evaluation carried out in 2012. The purpose of the pilot was to bring to rural areas private companies that were already operating in urban areas,shift the community management to a commercial operation that would increase the service quality and the number of customers, and bring external private funding for facility improvement to reach commercial viability and financial autonomy. The project ended up failing to attract skilled urban companies. In addition, the non-competitive selection of operators and the inappropriate contractual arrangements did not set up effective incentive mechanisms. Although some improvements were found in the performance of the local water service, the evaluation concludes that the pilot was not a success. It recommended refining the model by building on the experience of other countries and donors.
Irrespective of the management model, appropriate governance, monitoring, regulation and accountability mechanisms need to be in place at all levels to support the sustainability of the water service. Evaluations found little transparency in the management of O&M funds as well as inexistent governance and accountability mechanisms in a majority of UNICEF intervention communities. There seems to be a lack of both ability and willingness from local committees or operators to engage in this direction, although good management and transparency usually trigger a virtuous circle by building trust, increasing tariff collection rates and improving user satisfaction. A lack of governance is also found between the committee or operator on the one hand, and the local authority or the national agency on the other. This lack of governance is due to weak initial involvement of the local authority when the infrastructure is built, low level of subsequent interest and oversight, and lack of capacity of the supervisory body. The Burundi 2013 report also points out that the “mismatch with concepts of local ownership and decentralisation” is used by both parties as a pretext to disengage from some of their respective duties. The UNICEF/UNDP joint WASH programmes in Bosnia Herzegovina and the Philippines are the only two programmes that have significantly invested in developing effective governance mechanisms at local level through institutional reforms and tailored capacity building. More upstream work is also needed in order to establish or strengthen performance monitoring and regulation systems at national level, with effective incentives and sanctions. The absence of such systems as possible cause for unsustainable water services seem to be addressed neither in evaluations nor in UNICEF WASH programming, and this is one of their major weaknesses. “Policy level work needs to be tackled more forcefully, with sufficient time and resources allocated to achieve greater effectiveness and improve chances of sustainability” (Bosnia Herzegovina 2013).
Contextual and environmental factors influence sustainability in various ways amply developed above. Threats that are more specific to water supply interventions and service are corruption – because of the cost of drilling programmes and the fact that water services generate revenues – and climate change that affects the service functionality, all-year availability and water quality.
Sanitation and Hygiene (63 reports from 37 countries)
This subsection will focus first on CLTS which is the most common sanitation and hygiene intervention in UNICEF, and more particularly on the presence and use of latrines. Handwashing and other hygiene interventions/behaviours will then be discussed further below.
Approximately ten evaluations and sustainability checks were carried out one year or more following the ODF certification. The exact number is not known as a number of reports fail to specify how much time elapsed between the certification and the field re-verification. These reports provide a good indication of the level of sustainability of CLTS interventions as implemented by UNICEF in the field. Nevertheless, it should be noted that a few reports do not quantify their finding. Moreover, the robustness and comparability of quantitative evidence varies a lot depending on the definition of ODF used (or not used) in the report, the data sources, and data collection methods and analysis.
Numerous, slightly different sustainability indicators can be found in the reports: Percentage of certified communities that still meet the ODF conditions (depending on the nationally agreed definition of ODF); Percentage of certified communities where no evidence of open defecation can be found; Percentage of households living in these villages and that still have a functional, private latrine (they must sometimes meet some additional requirements such as a solid platform, a cover etc.); Percentage of households having a functional private latrine with a functional handwashing device next to it; Percentage of households using a latrine that is not necessarily their own private latrine: this include shared and public latrines for example; Percentage of household having re-built their latrine after they have been damaged etc. Data are usually collected through structures observations in and around a sample of villages as well as in a subsample of households, looking for evidence of open defecation from human beings and/or assessing the existence and the condition of latrines and handwashing stations. These data are sometimes triangulated through structured interviews with household members and implementing partners’ field staff. Some reports over-rely on interviews without triangulating with direct observation.
Globally, the proportion of villages where visited household still had functional latrines and where no evidence of open defecation were found a certain amount of time after certification ranges between 50% and 100%. In Mali (2014), an endline survey conducted six to 19 months after the ODF certification found that “the proportion of households with access to a private latrine was not associated with [any reversal of behaviour toward open defecation in the] months since the village received ODF certification”. In Timor Leste (2015), an endline survey conducted in some cases up to three years after certification found that 20% of villages reverted to open defecation, but the final evaluation did not confirm it: it found only 8%. In the Ethiopia 2010 evaluation, no evidence of open defecation was found but there were indications that some of the latrines were not being used. Two years later, another evaluation found that one to five years after ODF declaration, 96% of the latrines visited were functional and used, however signs of open defecation were found in 18% of cases. In Zambia (2012), the sustainability check found serious departure from ODF status in 20% of the sample, one year or more after certification. In Mozambique where a sustainability check has been carried out every year since 2009, the percentage of households not having a functional latrine anymore was 1% one year after certification and 22% after four years. The global CATS evaluation (2014) consulted CLTS implementers and concluded that less than 25% of households revert back to open defecation. The highest rate of slippage was found in Madagascar (2014) where field observations showed that more than half of the villages did not meet the ODF criteria any longer (time since certification not specified). The overall average rate of slippage seems to be around 20% one to five years following certification. This is consistent with the finding of the main robust ODF sustainability studies carried out recently by WSP (2011) and Plan (2013). The WSP study found that 95% of households were still using improved latrines 4.5 years or more after ODF certification. The Plan study found, two or more years after ODF declaration, a rate of 13% based on the observed presence of a functioning and used latrine, and 26% based on both the presence of a functioning and used latrine and the absence of excreta in the vicinity of the house. It is noteworthy that the above results were found in contexts where little to no reinforcement activities had been carried out since certification. These sustainability rates should also be compared to the ones of non-CLTS programmes. A few UNICEF evaluations of rural sanitation programmes carried out before the introduction of CLTS (Kenya 2009, Burkina Faso 2012 and Mozambique 2014 referring to earlier data) or in countries that still use other approaches (DRC 2014, Pakistan 2014 where demonstration latrines are still built) suggest that the proportion of damages and abandoned latrines is similar if not higher, which is consistent with the findings from the WSP 2011 study in Bangladesh.
As pointed out in the global CATS evaluation (2014), the key question is not whether there will be cases of slippage within certified communities – this will always happen either because latrines get old and damaged or because newcomers settle down in the community for example – but “how these cases are dealt with and what is the critical number of defaulters above which the social norm is put at risk”. According to the Madagascar evaluation, CLTS leaves its mark in the communities and in people’s mind-set: “They are not tolerant vis-à-vis open defecation but vis-à-vis individuals that practice it. Interviews prove that people judge open defecation much more harshly than before the introduction of CLTS in their village”. The Cambodia 2009 evaluation report highlights that the slippage rate greatly varies throughout the year, approaching 100% during heavy rains and significantly reducing again in the dry season after people have re-built their latrine. So the evolution of the ODF status over time is not linear. The report also emphasises that 0% open defecation is almost never reached since some people continue to defecate in the open when working in the fields. The common practice of re-building the latrine as a result of effective change in behaviour and social norm is also documented in other reports (Mauritania 2010, Sierra Leone 2013, Mozambique 2009, 2013 and 2014 etc.). Longer-term evaluations would shed light on how long improvements in latrine use persist through time. Unfortunately, sampling methods and data collection and analysis used in sustainability checks are yet not robust and comparable enough to draw meaningful conclusions.
The reasons for sustainability and slippage are being increasingly documented in the literature. They are numerous. UNICEF evaluations and sustainability checks discuss some of them, but not all. They are examined below using the same categories of sustainability determinants as above.
With regards to the technical factors, the fact that the CLTS approach encourages households to use their own construction material and techniques without depending on external skills or in-kind and financial inputs makes it more sustainable. They are able to easily rebuild for themselves or help newcomers whenever necessary. The downside of this approach is the poorer quality of the latrines as stressed in all reports. The pit (with or without lining) and the slab are the most fragile elements, due to the design, technique and material used. Environmental conditions such as rocky or sandy soil, heavy rains, floods, high water table or termites, can make latrines collapse. As a result, many of them need to be repaired or rebuilt every year or every few years. The shallowness of some pits is also an issue as they will have to be emptied (or rebuilt) more frequently. The reports reviewed do not provide quantitative data on the evolution over time of the proportion of latrines that need to be repaired, emptied or rebuilt, and the frequency. Besides the quality of latrines, the other technical aspect that influences sustainability is the availability, accessibility, appropriateness and attractiveness of sanitation materials and services needed to maintain, repair or rebuilt the latrine. Water, wood, mud or bricks may not be easily available, or they may not meet households’ demand. Masons may be located far or not have the appropriate skills. In some areas, the lack of space in the household can hinder the reconstruction of a new latrine. The use of shared latrines tend also to be a sustainability issue. They are more difficult to manage and tend to quickly become dirty, enticing users to revert to open defecation. It should be mentioned however that these three technical issues are not specific to CLTS. They are also mentioned in the non-CLTS evaluations listed above. Technical obstacles to the relative sustainability of ODF interventions may well be over-estimated if people still manage to rebuild their latrines as suggested by the slow decrease in the ODF rate over time and evidenced in the Cambodia and several other reports.
The global CATS evaluation (2014) states that “in countries where the challenge facing households around durable materials for latrine construction is widely acknowledged, UNICEF’s efforts have been more oriented towards ensuring that improved traditional latrines are acceptable within national standards than in making durable materials for latrine construction more locally available”. Some evaluations found no engagement with supply chains and the private sector at all. Communities are left to organize themselves and reach out to whatever skills they can find locally, in line with the initial principles of CLTS. In Nepal (2013), it was mentioned that the “local private sector’s role grew critical in making physical infrastructure available for toilet installations, overcoming financial challenges at the local level, and in distribution system by making materials available easily and efficiently”. Nevertheless, in several countries UNICEF began to address technical issues by mobilising and capacitating the local private sector. In some of them, UNICEF trained local masons, provided them with tools and supported the creation of local ‘sani-marts’ (Ghana 2009, Zimbabwe 2011). In others, more recently, UNICEF piloted ‘sanitation marketing’ as a way to support the design, fabrication and marketing of durable latrines / latrine materials at a regional or national scale (Indonesia 2013, Timor Leste 2015). A few evaluators found that sani-marts and sanitation marketing pilots have been generally unsuccessful so far due to the lack of financial viability and the attrition of trained masons, or limited in scale (Ghana 2009, Indonesia and Nepal 2013, Pakistan 2014, Timor Leste 2015). However, more evaluations required in the coming years to assess the relevance and effectiveness of these respective approaches as well as the conditions of success.
Financial determinants are at play at both household and national levels. At household level, the limited capacity of household to pay for latrine maintenance and improvement is systematically cited as a major obstacle to sustainability by evaluators, based on their own perception and on interviews with beneficiaries and implementing partners. Rural households living in small villages are poorer than other households. Those headed by women, elderly or disabled people, and those with a high number of children tend to have a lower income level. However, this financial obstacle may be over-estimated as well, for four main reasons. First, as stated in the Timor Leste 2015 evaluation, “observations shows that many poorly maintained latrines belong to families with much better constructed and maintained homes and other outbuildings”. Second, no or very little cash is needed to rebuild the latrine when using local material and labour. It is only needed in the case where a household wants to upgrade the latrine. Third, the limited capacity of households to pay for sanitation is the same argument that was used against CLTS when it started to be known and promoted by certain NGOs in the late 1990s and early 2000s. In practice, this approach still proved effective in getting millions of households to build latrines using their own resources. Fourth, all households have some kind of housing and there is no reason they could not regularly maintain their latrines the same way.
UNICEF addresses this potential financial obstacle in only a small number of countries, by putting in place financial instruments aiming to support households’ investment capacity. The village savings and loan associations in Sierra Leone are one example. Credit schemes were introduced in Ghana (2009). Pakistan (2014) decided to revert to the household subsidy approach and distribute vouchers to help poorer household build better quality latrines. Again, more evaluations would be required in the coming years to assess the relevance and effectiveness of these respective approaches.
Another financial aspect is the ability of governments and their development partners to finance reinforcement activities in certified villages and at national level (integration of sanitation in the work plan of community based health workers, post-certification monitoring visits, capacity building, mass media campaigns etc.). Some UNICEF programmes support it by mainstreaming CLTS as a national strategy for rural areas and by encouraging governments and other donors to commit funds for CLTS specifically. Examples for success are given in the scalability section above (section 5.5).
Other determinants are mentioned in many evaluation reports but reportedly less understood and taken into account in UNICEF programming: these are the social, cultural and psychological factors which, at various levels in the community, support or hinder the effectiveness and sustainability of CLTS. They influence the willingness and ability of the community, households and individuals to continue their efforts and maintain the ODF status after the certification. For instance, the willingness and ability of individuals and households is conditioned by the size of the household: the availability of manpower to dig the pit and build the latrine helps, while too many household members could lead to latrine overuse and maintenance challenges; it could necessitate to (re)build more than one latrine to avoid reversion to open defecation. The sex of the head of household may also play a role, with women being more engaged in sanitation issues than men. However, women-headed households have usually less resources (material, financial, workforce). The level of awareness and understanding of sanitation and related risks and benefits and, in general, the level of education, drives the level of commitment. Residential status and geographical mobility are equally important because, as pointed out by the Cambodia 2009 report, when people spend much time outside of the village for work, they use their latrine less and dedicate less priority and time to sanitation. Seasonal migrants and nomadic households also have less interest in investing in private sanitation facilities. In some rural areas where households rent their house, the readiness to invest in sanitation is lower. By contrast, having someone in the household with reduced mobility (elderly, disabled, chronically sick…) can motivate to keep or rebuild the latrine. There is little UNICEF can do when these facts become obstacles, besides taking them into account as criteria in the selection of beneficiary villages (e.g. avoid nomadic communities, villages close to towns and peri-urban areas) and letting implementing partners and field facilitators manage household level issues (large sized household and people with special needs).
At the community level, the involvement and contribution of the whole community is one of the elements that make CLTS more sustainable than previous approaches. These words are even too weak since most of the process is actually directly led by the community. However, some communities show more leadership and ownership than others. The understanding, participation, commitment and dynamism of the village sanitation (or WASH) committee, natural leaders, women and youth associations, teachers, children, community health promoter, village chief and religious leader are mentioned in several reports as an essential condition for ODF sustainability. The question of leadership in particular is discussed in several reports (Ghana 2009 and 2012, Mozambique 2013, DRC 2014) because in many rural villages, the traditional and religious chiefs have a strong influence on people’s decisions and behaviours. They are key in facilitating smooth communication, social cohesion and cooperative action which, in turn, drive the realisation of a public good such as sanitation and health. They can set rules and apply sanctions against the practice of open defecation. A number of reports evoke such by-laws and consider them as key when not leading to strong coercion and exclusion (WCARO 2010, Zambia 2011, Sierra Leone and Nepal 2013, Madagascar and Global CATS evaluation 2014…). Compliance with ODF rules is also ensured by internal peer pressure, from family members, neighbours, other community members, or village sanitation committee. Peer-pressure is one of the benefits from the community-led approach and specificities of CLTS.
UNICEF’s action to support community level engagement usually includes the involvement of national leaders, women (as natural leaders and members of the village sanitation committee) and children. In some countries, UNICEF also involves the local leaders and provides training to the village committee but it is not the case in all CLTS programmes. Specific involvement of women and youth associations, teachers and community health promoters is even less documented although they could be used as resource persons, leaders and powerful agents of change in their community.
When most or all leaders and community members begin to share the same aspiration, a new social norm is established and replaces previous cultural beliefs and practices that may have threatened the sustainability of ODF. Reported examples of such beliefs are: latrines are a ‘snobbish’ or ‘white man thing’; pits are potential sources of diseases; the underground belongs to ancestors and should not be used to dig latrines; defecating in the open reinforce vitality etc. The new social norm reflects and guides what people think others do, how they think other people expect them to behave, and how they will ultimately behave going forward. There certainly are preconditions and actions that enable the emergence of a new social norm. For example, most CLTS programmes only intervene in small communities with concentrated housing as it is assumed to be associated with greater internal communication, cohesion and leadership, and therefore favourable to a quicker shift in social norms (Global CATS evaluation 2014, Mali 2014, Timor Leste 2015…). The feeling of pride can emerge at both individual and community levels, when interacting with others. Nurturing this feeling has been identified as an effective strategy to maintain the momentum in ODF communities, because many people aspire to remain a model for others or keep a certain reputation. UNICEF has been reported to do it in Ghana (2012) and in a few other countries. But it is far from being systematically mainstreamed in UNICEF’s approach to CLTS. While a number of CLTS evaluations marginally deal with the topic of psychological, cultural and social determinants of CLTS sustainability, they do not bring robust evidence to the discussion. On-going studies within and outside UNICEF suggest that due to the community-led approach, these determinants may well operate in favour of sustainability rather than against it. However, it is clearly a new area of investigation for both WASH evaluations and UNICEF programming.
Institutional determinants embrace the mainstreaming of CLTS in sector policies and strategies, in governments’ budget and among donors; governments’ ownership at all levels; the level of coordination among development partners; the avoidance of concomitance of conflicting rural sanitation approaches (e.g. CLTS vs. subsidisation programmes); the capacity and engagement of ministerial departments at all levels and of local authorities; the existence of guidance and tools for reinforcement activities. As already developed above, UNICEF’s efforts in these domains have been tremendous and successful. Capacity building of public institutions and local authorities remains a continuous challenge. Training activities could be better designed and implemented. But the greater amount of work that remains is certainly on the institutionalisation of incentives at all levels: ODF communities, local authorities and government’s technical departments. These implies establishing rules, verification procedures, as well as benchmarking, rewards or sanctions – all ideally funded by national budgets.
It is obvious from the above that the quality of programme design and implementation augments the prospects for sustainability. Other examples are provided in evaluations. The DRC 2014 sustainability report and others demonstrated that concentrating CLTS activities in a well-defined geographic area eases contacts and reinforcement between neighbouring villages, helps create a ‘regional norm’, and makes regular post-certification monitoring visits logistically and financially possible. Village accessibility was also found to be associated with the sustainability of the ‘healthy’ status. On the negative side, the triggering of too many communities simultaneously can endanger sustainability if capacity constraints make it difficult to maintain the required level of mentoring and monitoring needed after the ODF certification. A poorly conceived theory of change also endangers sustainability, for instance when CLTS is implemented in disconnection with water supply or WASH in schools. As a matter of fact, a community with inadequate access to water will be less interested and capable to maintaining its ODF status. Similarly, a community that requires from children that they adopt good sanitation and hygiene behaviours without providing them with adequate facilities and education at school may experience higher rates of slippage. The quality of each step of the CLTS process itself is crucial. At the pre-triggering phase, eligibility or prioritisation criteria are used to enrol villages depending on the population size, latrinisation rate, housing density, accessibility etc. Based on the above analysis, more criteria could be used to maximise the likelihood of sustainability. The quality of the triggering process (broad and inclusive participation, facilitation skills, comprehensiveness and succession of all activities, visual exercises, clarity of messages, competition with neighbouring communities and public pledging…) and post-triggering monitoring (frequency of visits, empowerment of support provided to the sanitation committee, frequency of internal meetings, participation of community health promoters etc.) are also said to be instrumental not only in reaching the ODF status but also in maintaining it (Ghana 2009, Zambia 2011, Ethiopia 2012, Sierra Leone 2013, Bangladesh and Global CATS evaluation 2014, Timor Leste 2015 etc.). The existence of significant external rewards to build latrines or reach the ODF status may jeopardise sustainability. They can become an “incentive scheme” or even a type of output-based subsidy influencing communities to move towards quick results rather than adopting more durable, embedded behaviour change (Mozambique and Sierra Leone 2013, Global CATS evaluation 2014, Timor Leste 2015 etc.). Quality of certification can be affected the lack of rigorous verification procedures and observation checklist leading to communities being certified when they should not have. A certain duration between ODF self-declaration and actual certification may help the community maintain their efforts but when too long, it may demotivate the community. These programme quality issues are very commonly found in evaluation reports, with countries performing better than others in some areas. In average, each of these above listed issues is mentioned in one-third of the reports. Cases of good practices were identified. For instance Sierra Leone seems to have a very thorough strategy in the implementation of the phase between triggering and certification. Regular monitoring visits are undertaken by field partners, local officials, UNICEF staff and the regional coordination task force, alongside radio panel discussions, bi-monthly meetings of natural leaders, and cross-community exposure visits between natural leaders. The Global CATS evaluation regrets that good practices are rarely documented and disseminated within UNICEF WASH staff.
The post-certification phase gained much attention in the past years although earlier evaluations were already underlining the need for reinforcement activities (Ghana 2009, Ethiopia 2012 etc.). Because this phase became a concern of WASH teams only recently, most evaluations under review criticised its weakness. In some programmes, reinforcement activities include: regular monitoring visits by implementing partners for a few months after ODF certification (longer in the case of Mozambique); introduction of new sanitation and hygiene related messages on grey water drainage, solid waste management and drinking water treatment; improvements on the water source; provision of additional training to villages committees and masons; launch of a pilot or full sanitation or WASH marketing programme. Unfortunately, potentially impactful activities such as large and repeated media campaign using multiple communication channels, regular visits by health extension workers and local and government authorities, institutionalisation of supervision and incentive arrangements, integration of sanitation and hygiene themes in healthcare priorities and indicators, and creation of new national monitoring indicators on ODF sustainability have not been reported. Regarding the later, periodic, government-led sustainability checks could become such an ODF monitoring process if applied with a robust methodology.
Most of the environmental determinants affecting sustainability have been presented above. One specific remark with reference to CLTS is that climate change, drought and desertification increase the distance between the houses and the bush previously used for defecation and thus may motivate people to use their new latrines. At the same time, depleting natural resources are a source of raw material for latrine (re)construction.
In many countries, UNICEF used to promote handwashing as a separate programme component. It is now integrated in CLTS programming as one of the ODF criteria. Unfortunately, this shift does not seem to have achieved better results in terms of effectiveness and sustainability of behaviour change according to evaluations and sustainability checks. In both intervention strategies, the handwashing rate ranges between 5% and 25% at least one year after the intervention (Ghana, Guinea Conakry and Guinea Bissau 2009, DRC 2009 and 2012, Malawi and Zimbabwe 2011, Ethiopia, Malawi and Kenya 2012, Mozambique and Zambia 2013, DRC, Nigeria, Mozambique, Bangladesh, Mali and Madagascar 2014 etc.). These data based on observation of functional handwashing stations with water and soap or ash in the household or near the latrine are remarkably consistent across countries and reports. Data based on self-report, used in several reports, are much higher and biased.
The sustainability of drinking water treatment interventions has been discussed in three reports. The Malawi 2009 report found that nearly a third of participants reported purchasing and using the WaterGuard treatment product after receiving their free bottles, suggesting that water treatment behaviours motivated by the programme may be sustainable after the free products are distributed. The same year, another evaluation conducted in Guinea Bissau and Guinea Conakry with a more robust methodology revealed that in both countries people still understood that chlorine could protect them from cholera and remembered how much chlorine to apply to their drinking water. But in Guinea Bissau they also indicated that they would only apply it during the cholera season or when the first cases of cholera were declared within their community. This sustainability issue was confirmed by high rates of drinking water contamination within the households. In Guinea Conakry the actual use of the product was better but still lower than expected, with only 40% of households practising daily use. In Liberia (2012), no evidence of sustainability in household water treatment practices was observed after the free distribution of WaterGuard in the initial phase of the intervention.
Just like CLTS, handwashing and water treatment promotion are behaviour change interventions. Although they involve a much less significant hardware component, they face the same type of sustainability issues: technical; psychological, cultural and social; financial; institutional; contextual; and issues related to programme design and implementation. Evaluations do not provide a detailed analysis of these determinants and rarely attempt to assess UNICEF’s performance in addressing them. The first lesson learnt from the reports is that the level of success of door to door visits and mass media campaigns varies a lot from country to country and none of them alone is sufficient to achieve sustained behaviour change. A multiplicity of communication channels must be repeatedly used over a long period of time. The second lesson learnt is that UNICEF field staff not only has limited expertise in the subject matter but also rarely tries to mobilise external expertise and commission detailed formative research which could guide the design and implementation of the hygiene promotion programme. Yet, “sustainable use of disinfection products and hand washing with soap can only be achieved by understanding why people practice or not practices certain behaviours and what could motivate people to change these behaviours” (Guinea Bissau and Conakry 2009). There is certainly much to learn by UNICEF from its experience with CLTS that could be applied to hygiene promotion programmes. It is noteworthy that in six CLTS and two hygiene promotion evaluations health benefits were cited by beneficiaries as main motivator for behaviour change. This contradicts the literature that emerged in the late 2000s stating that health is seldom among the most important reasons for behavioural and purchasing decisions of hygiene and sanitation consumers.23
These somewhat disappointing findings on sustainability of hygiene promotion programmes and practices are partly counterbalanced by the findings from the Senegal 2008 report. This evaluation was conducted ten years after the implementation of a combined education, protection and hygiene programme in three regions of the country. The purpose of the evaluation was to assess the long-term sustainability of this intervention. Results suggest that handwashing and water treatment may be the most difficult hygiene behaviours to change. Nevertheless hygiene promotion programmes may have more positive and lasting effects on other individual and group dynamics that are not captured in classic programme evaluations: “The programme seems to have promoted a ‘culture of hygiene’ in the villages. Cleaning now seems to be a precursor to any action. ‘Tostan taught us how we can maintain our children, our household and ourselves in hygiene and cleanliness, but also informed us on the hygiene and cleanliness of our food.’ During the evaluation, we could not find functional committees besides those that worked on hygiene. Only the actions of the hygiene committee are still noticeable. In conclusion, we can say that the programme reinforced the knowledge of most of the women surveyed, and gave them a better grasp of health and hygiene information. (…) The programme also helped increase women’s confidence in their own capacities and their leadership aspirations. Hygiene and health are the areas where the lessons learned had a very quick effect”.
