This report presents the findings of the programme evaluation on HIV/AIDS commissioned by the Dutch Co-Financing Agencies Cordaid, Hivos, Icco, Novib and Plan Netherlands. It is the fifth study in a series of programme evaluations organised during the period 2003-2006 by the MBN, the Network of Co-Financing Agencies (CFA’s) in the Netherlands. The evaluation was carried out by ETC, Crystal (first phase) and ACE-Europe (second phase) in 2005.
The HIV/AIDS epidemic continues to spread and erode development gains, not just in Africa, but in many parts of the world. In assessing the CFA’s response to HIV/AIDS there are certain challenges. First, HIV/AIDS is a new and unprecedented disease—one that spreads silently, invisibly and rapidly, and then kills 10 years later. Secondly, it kills the most productive people of society, young people and adults between 25-45 years old. The impact extends to society at large and seriously threatens social fabrics and economic development. The learning process about the exact impact of HIV/AIDS and how to mitigate this, is ongoing. A third challenge is that HIV/AIDS remains incredibly sensitive to discuss, and those infected are often stigmatised. This is both because HIV is spread sexually and through injecting drug use and because it is nearly always fatal in developing countries. A fourth important challenge is the feminisation of the HIV/AIDS pandemic. An increasing proportion of people infected by HIV/AIDS is female. Girls and women everywhere are highly vulnerable because of sex – and gender-based factors.
This evaluation reviews the CFA’s response to the HIV/AIDS pandemic from 2001 onwards and assesses the effectiveness, efficiency, relevance and sustainability of the CFA’s assistance. In the 1990’s early CFA action on HIV/AIDS involved support for AIDS focussed organisations from an emancipation point of view (HIVOS) or support for the health sector to integrate issues of HIV/AIDS (Cordaid). From 2001 onwards, besides supporting AIDS focussed organisations, the CFAs started to make a commitment to HIV/AIDS mainstreaming. The CFAs have since developed different strategies for addressing the problem in all regions and a variety of sectors. The priority given to HIV/AIDS has markedly increased. In the evaluation period (2001-2004), the CFA funding almost doubled: from Euro 12 million in 2001 to Euro 22 million in 2004.
Considerable progress has been made since 2001 by the CFA’s. HIV/AIDS related positions have been created, HIV/AIDS working groups or teams and focal points put in place, resource materials developed, new partnerships established, trainings organised, funding provided for HIV/AIDS programmes. Since 2003, the CFA’s have developed innovative approaches to internal mainstreaming and also started interesting experiments and research on external mainstreaming. The AIDS-focussed organisations visited are almost all involved in the delivery of comprehensive package of HIV/AIDS services. The strengths of their programmes lies in the holistic approach to HIV/AIDS, combining and linking HIV/AIDS awareness, voluntary testing and counselling (VCT), Home Based Care (HBC), orphans and vulnerable children (OVC) and income generating activities. The generalist organisations visited acknowledged the importance of integrating HIV/AIDS into their strategies and programmes. They responded to the HIV/AIDS epidemic adopting or modifying their programmes to the needs of the beneficiaries. They took the specific needs of PLWHA and affected households into account. The majority of the organisations ‘added’ additional HIV/AIDS-related activities to their current programme, only a few modified their core activities to respond to the changing context, exploring in depth the link between their programmes and HIV/AIDS. This resulted in an improved access of their constituencies to HIV/AIDS related services.
The CFA’s will give follow-up to this evaluation in several ways:
A public presentation and discussion will be organised to both share the findings and recommendations. The CFA’s expect to attract the interest of a broad public: CFA staff, including managers and directors, staff of other Dutch development organisations and academic institutions, and the Ministry of Foreign affairs;
Each CFA will carefully reflect on the lessons learned and the recommendations of this evaluation and design an internal follow-up process: an action plan with budget, planning activities for the CFA and for its partners;
Each CFA will take the recommendations into account, when developing their new strategic business plans for 2006-2010.
Where relevant and as much as possible the CFA’s will work together through SAN!. This collaboration can range from joint projects to lobby and advocacy;
The CFA’s learned from this evaluation to continue with some successes: being flexible donors, which is important in times of HIV/AIDS because the pandemic brings a lot of uncertainty to partner organisations and communities; emphasising HIV/AIDS workplace policies for local NGO’s; increased funding; organising linking and learning sessions and other capacity building processes; facilitating mainstreaming of HIV/AIDS.
The CFAs also learned that they need to improve on the following: more advocacy to include gender-based factors in HIV/AIDS policy-making and programming; challenging partners to include boys and young men in HIV/AIDS prevention activities in order to change heterosexual behaviour norms and values; challenging faith-based organisations to promote evidence based prevention methods, e.g. condoms; giving more and thorough follow-up to training and linking and learning activities; developing tools to translate policies into practices, pre-test them and make sure staff uses them; developing action plans, including budgets for boosting the important mainstreaming processes in existing partners and building HIV/AIDS competence of CFA staff.
The CFA’s would like to thank all those who have been involved in this evaluation for their co-operation, especially the partner organisations and the country and regional offices and departments. We would also like to acknowledge the work done in the first phase of the evaluation by Ms. Joanne Harnmeijer (ETC Crystal) for exploring the outline of this evaluation. We express our sincere appreciation to the way Mr. Stan Bartholomeeussen and Ms. Geert Phlix (ACE-Europe) designed and conducted this evaluation. The important technical advice from the MBN Reference Group, as well as the internal support of the CFA Co-ordination Group is gratefully acknowledged. Lastly, the views expressed in this report are those of the ACE-Europe evaluation team. We hope you enjoy reading the report.
28th of February 2006
The MBN directors:
Sylvia Borren, Novib
Jack van Ham, ICCO
Manuela Monteiro, Hivos
Paul Lem, Plan Netherlands
René Grotenhuis, Cordaid.
“…the AIDS pandemic is exceptional as a global crisis and threat. This pandemic is exceptional because there is no plateau in sight, exceptional because of the severity and longevity of its impact, and exceptional because of the special challenges it poses to effective public action….an exceptional response is needed.” (Peter Piot, executive director UNAIDS, 2005)
The team succeeded in the effort to organise the visits to the different countries well and wants to thank everybody that made this possible, in particular, all the counterparts of the 5 CFAs that welcomed the team in their organisation (often at very short notice) and organised the visits to the beneficiaries. Without their very flexible way of dealing with our visit, the evaluation missions would not have gathered the important evidence material we can now use for responding to the different evaluation questions. The team also appreciated the time invested in this evaluation by all HIV/AIDS policy officers and their colleagues!
WASN Women AIDS Support Network
AIDS focussed organisation an organisation that has as core business the fight against HIV/AIDS and that conduct programmes and activities (including lobby-activities) related to prevention, treatment, care or support for those infected.
Generalist organisation an organisation that has as core business (local) development or humanitarian action. When an organisation installs a separate unit that is specialised in AIDS work, the consultant consider this (newly created) organisation as an “AIDS focussed organisation” (and not as integrated AIDS work) when they found evidence of the fact that this organisation works independently from the “mother” organisation, could contract independently and be self sustainable.
AIDS work work directly focused on AIDS prevention, or care, treatment, or support for those infected. Work that is distinct and implemented separately from other existing development and humanitarian work.
Integrated AIDS work AIDS work which is implemented along with, or as part of, development and humanitarian work. The focus is on prevention, care, treatment, or support but with the difference that the work is conducted in conjunction with, and linked to, other projects or within wider programmes.
Internal mainstreaming changing organisational policy and practice in order to reduce the organisation’s susceptibility to HIV-infection and its vulnerability to the impacts of AIDS. The focus is on AIDS and the organisation. It has two elements: AIDS work with staff, such as HIV prevention and treatment, and modifying the ways in which the organisation functions.
External mainstreaming adapting development and humanitarian programme work in order to take into account susceptibility to HIV transmission and vulnerability to the impacts of AIDS. The focus is on the core programme work in the changing context created by AIDS.
Work place policies formalise an organisation’s position with regard to HIV and AIDS and its staff. Policies usually cover provision of awareness raising and counselling for staff, but should extend beyond those measures to a range of human resources or personnel issues, such as staff health schemes, arrangements for sick leave and compassionate leave, and terms and conditions of employment with regard to HIV and AIDS.
AIDS work with staff activities to help staff to face up to AIDS as a personal issue (questionnaires, workshops, involving PLWHA in training and workshops, discussing internal mainstreaming, etc…), staff awareness and education (from a gender and sexuality perspective) and prevention activities for Staff (availability of condoms, VCT, etc.).
HIV/AIDS competence An AIDS-competent society means that people in families, communities, organisations and in policy making acknowledge the reality of HIV/AIDS and act from strength to build capacity to respond, reduce vulnerability and risks, learn and share with others and live out their full potential. (source: the constellation for AIDS competence)
SAN! Stop AIDS Now is a joint initiative of NOVIB, CORDAID, HIVOS, ICCO and the AIDS fund to join forces in the fight against HIV/AIDS. At the beginning SAN! was focussed on public awareness building and fundraising. Since 2004 SAN! has become also active in lobby and advocacy work and the CFAs have started joint research development projects within SAN!.
Table of content
1. Introduction 16
1.1. Context and purpose of the evaluation 16
1.1.1. Context of the evaluation 16
1.1.2. Objectives of the assignment 17
1.1.3. The evaluation team 17
2. Methodology 19
2.1. The elaboration of the four evaluation questions 19
2.2. Process of responding to the four evaluation questions, based on the data collection in the field 21
2.3. Sample of cases that have been visited 23
2.4. Constraints faced during the execution and limitation of this evaluation assignment 25
3.1. The HIV/AIDS context in which the CFAs are operating 28
3.2. Description of the HIV/AIDS policy of the five Co-Financing Agencies 32
4. Assessment of the Evaluation questions 38
4.1. To what extent did the CFAs respond to the HIV/AIDS epidemic and build up competence to cope with HIV/AIDS? 38
4.1.1. State of affairs in relation to policy development and policy implementation 38
4.1.2. Assessment of achievements 58
4.2 To what extent and how successfully did each CFA perform in enabling the different counterparts to contribute to the fight against HIV/AIDS and to cope with the HIV/AIDS epidemic? 64
4.2.1. State of affairs in relation to the three judgement criteria 65
4.2.2. Assessment of the achievements 76
4.3. What has been the counterparts’ contribution (AIDS-focussed organisations) to the fight against HIV/AIDS ? 81
4.3.1. State of affairs in relation to the three intervention strategies of the CFAs 82
4.3.2. Assessment of the achievements 94
4.4. To what extent did the counterparts (generalist organisations) cope with HIV/AIDS ? 100
4.4.1. State of affairs in relation to the three judgement criteria 101
4.4.2. Assessment of the achievements 109
5. Conclusions 115
5.1. Conclusions on CFA’s HIV/AIDS competence (EQ 1) 115
5.2. Conclusions on the partnership relation (EQ 2) 118
5.3. Conclusions on the performance of the HIV/AIDS focussed organisations in Southern Africa and India (EQ 3) 120
5.4. Conclusions on the performance of generalist development organisations (EQ 4) 121
5.5. Conclusions on the overall evaluation question 122
6. Recommendations 124
6.1. Increase HIV/AIDS competence within the CFAs 124
6.2 Establishing real partnerships 125
6.3. More collaboration at different levels 125
6.4. Implementing the donor guidelines for Workplace policy and looking for local solutions 126
6.5. HIV/AIDS focussed organisations versus generalist organisations 127
6.6 Increase focus on gender 128
7. Annexes 130
Executive summary Five Co Financing Agencies (CFAs: HIVOS, ICCO, Novib, Cordaid and Plan) participated in the MBN HIV/AIDS evaluation. The evaluation assessed the relevance, effectiveness, efficiency and sustainability of the CFA’s funding strategies, policies and practices in the fight against HIV/AIDS in the Southern African and South Asian regions. The CFAs wanted to know the extent to which their support in the period 2001-2004 has contributed to the fight against HIV/AIDS. The intervention strategy of the five CFAs comprises (1) poverty reduction, (2) civil society building and (3) advocacy and lobby. In the context of HIV/AIDS, “poverty reduction” can be seen as the delivery of HIV/AIDS services, such as prevention, access to treatment, care and support initiatives. Civil society building is about the support to, and capacity building of emerging community-based organisations (CBO’s), peer support groups and NGOs involved in the provision of HIV/AIDS related activities. Advocacy and lobby concerns the support to CBOs and NGOs who are involved in advocacy and lobby activities on HIV/AIDS issues, or the participation in networks and campaigns at a regional and international level to advocate and lobby HIV/AIDS-related issues, mostly on the rights of PLWHA, access to treatment and increased funding for HIV/AIDS programmes.
The overall evaluation question was divided into four sub-evaluation questions:
(1) To what extent did the CFAs respond to the HIV/AIDS epidemic and build up competence to cope with HIV/AIDS
(2) To what extent and how successfully did each CFA perform in enabling the different counterparts to contribute to the fight against HIV/AIDS and to cope with the HIV/AIDS epidemic ?
(3) What has been the contribution of the HIV/AIDS focussed counterparts to the fight against HIV/AIDS ?
(4) To what extent did the generalist counterparts cope with HIV/AIDS ?
The evaluation included visits to 34 counterparts in four countries: South Africa, Malawi and Zimbabwe, which are considered as high HIV/AIDS prevalence countries, and India, which is considered a low prevalence country, but is experiencing a generalised epidemic in some states. Of each CFA, seven counterparts were visited, at least one counterpart per CFA per country. In each country, the sample contained a balance between AIDS focussed and generalist organisations. The evaluators held interviews with the management and staff of the counterparts, following the different judgement criteria. At each counterpart, the evaluation team spent at least one day in the field, interviewing and holding focus group discussions with beneficiaries.
The evaluators identified the following differences among the CFA’s:
Hivos focuses on HIV/AIDS specific organisations and its strong involvement in lobby and advocacy programmes.
Hivos and Novib prioritise support to prevention and awareness programmes, with Novib focussing on gender based violence and women’s rights and Hivos on sexual rights issues and specific target groups such as men having sex with men, migrants, refugees, sex workers, etc.
Cordaid and ICCO work among others with faith based organisations involved in the whole prevention to care continuum. Cordaid in particular has invested in treatment programmes and has supported some experiments. Cordaid and ICCO also challenge the position of the church regarding their prevention approach focussing solely on abstinence and faithfulness as well as their position regarding the rights of people living with HIV/AIDS.
Plan, as a child rights organisation, focuses on the support of orphans and vulnerable children, their families and communities. Plan supports programmes that cover the whole prevention to care continuum, including impact mitigation.
Considerable progress has been made since 2001 by the CFA’s. HIV/AIDS related positions have been put in place. Hivos created a specialised team of HIV/AIDS programme officers (vertical structure), Novib, Cordaid, ICCO and Plan created focal points and HIV/AIDS project groups or working groups (horizontal structure). Resource materials have been developed, new partnerships established, trainings organised, funding provided for HIV/AIDS programmes. Since 2003, the CFA’s have developed innovative approaches to internal mainstreaming and also started interesting experiments and research on external mainstreaming.
All CFAs have approved policies regarding HIV/AIDS (except Plan which is in a drafting process). The creation of HIV/AIDS working groups and teams and the appointment of HIV/AIDS policy officers have accelerated the development and implementation of such a policy. However, concrete annual action plans are often lacking and so is the systematic follow-up. The HIV/AIDS officer and the focal points do not have sufficient time available to always be on hand for the necessary support to build up adequate competence within the CFAs to cope sufficiently with the epidemic which is constantly changing. Knowledge management on HIV/AIDS is limited and that in combination with a sometimes frequent rotation of personnel, leads to a lack of HIV/AIDS competence by programme or country officers who often have a ‘generalist’ profile. Therefore, dialogue with counterparts on HIV/AIDS issues becomes difficult or remains superficial in the event of a monitoring visit. Apart from a few examples (related to Hivos and Cordaid), the evaluators did not notice an in-depth exchange on HIV/AIDS between the CFA programme officer and the counterpart.
The CFAs are rarely (individually) involved in HIV/AIDS related advocacy and lobbying at the international level and at the level of the Netherlands. The CFAs fund SAN! and Sharenet in the Netherlands as well as international networks or organisations for lobbying the European Commission, UN agencies or others. The evaluators consider this as an appropriate choice, taking into account that lobby expertise on HIV/AIDS-related issues lies within these networks and international organisations more than within the CFAs.
All counterparts in Southern Africa are aware of the importance of internal mainstreaming and are in the process of drafting a HIV/AIDS workplace policy. In India, the understanding of internal mainstreaming is generally poor, given the present stage of the epidemic. To support these processes the CFAs have challenged their counterparts through partner consultations and the organisation of workshops, mainly on internal mainstreaming (workplace policy development). The counterparts appreciate these efforts but are expecting a more challenging dialogue with their respective programme officers, in particular on the implementation of good practices.
The CFAs have demonstrated a certain degree of flexibility with regard to budget spending, for example through opting for budget support schemes. This enabled counterparts to adjust programmes and budgets according to the needs of their beneficiaries, which is essential in a rapidly changing HIV/AIDS context.Most of the counterparts visited appreciated the additional support (ad hoc budgets for internal mainstreaming processes) for the development of HIV/AIDS workplace policy, but feared that limited funding will affect implementation .The approved guidelines by the CFA’s within SAN! in December 2005 on ‘Support to Partners to Manage HIV/AIDS in the Workplace’ with financial commitments is an effective answer to these concerns. These guidelines are adopted for the two pilot countries of Uganda and India (except Novib who adopted them for all countries and counterparts).
The AIDS-focussed organisations visited are almost all involved in the delivery of comprehensive package of HIV/AIDS services. The strengths of their programmes lies in the holistic approach to HIV/AIDS, combining and linking HIV/AIDS awareness, VCT, HBC, OVC, income generating activities. A number of AIDS-focussed
Organisations are working on the enhancement of coping mechanisms of families, but in this they rarely looked for partnerships with experienced develop or generalist organisations with for example livelihood programmes. The support of CFAs towards the counterparts has resulted in an increased capacity of service providers and an increased number of beneficiaries. However, outputs and outcomes remain somehow limited due to the small scale of interventions, the similar approaches and the varying quality of services offered. The evaluators noticed in the interviews and focus groups discussions with beneficiaries, a good understanding of HIV/AIDS and how to protect themselves. However, many of the organisations do not seem to reach men to discuss responsible attitudes and male sexual behaviour. Concerning advocacy and lobbying, some counterparts are acknowledged as leaders at a national level in advocating for the rights of PLWHA and access to treatment. But the role of most counterparts remained at local level. This is due to the limited human resource and lobby capacities and missed opportunities to link their experiences and practices to national and international lobby networks.
The generalist organisations visited acknowledged the importance of integrating HIV/AIDS into their strategies and programmes. They responded to the HIV/AIDS epidemic adopting or modifying their programmes to the needs of the beneficiaries. They all aim to take into account the specific needs of PLWHA and affected households. The majority of the organisations ‘added’ additional HIV/AIDS-related activities to their current programme, only a few modified their core activities to respond to the changing context, exploring in depth the link between their programmes and HIV/AIDS. This resulted in an improved access of their constituencies to HIV/AIDS related services. The strategies of the generalist organisations are in general not based on a systematic needs assessment, but on their daily practice (except the JOPM programme did a HIV/AIDS community analysis). Generally, these external mainstreaming processes were initiated by the counterparts themselves, unlike internal mainstreaming, which were mainly donor-driven. External mainstreaming is felt by counterparts as a difficult task that needs careful knowledge management and increased support and funding.
Most of the counterparts are gender sensitive in their HIV/AIDS programmes, responding well but in a classical way to the needs of women by improving their access to a variety of HIV/AIDS services. However most of them do not contribute effectively to the empowerment of women, including the necessity to change the (power) relationship between men and women in a HIV/AIDS context. Addressing absolute poverty and all forms of societal violence that women and their children face are prerequisites to win the fight against HIV/AIDS. All HIV/AIDS policies approved by the CFAs subscribe the gender – HIV/AIDS relationship as being fundamental in their fight against HIV/AIDS. The evaluation team, however, did not find well drawn up strategy or action plans for the implementation of those policies (some exception related to programmes on gender based violence). The first initiative in that direction is the new SAN project “Gender, women and HIV/AIDS”.
The contribution of the CFAs to the fight against HIV/AIDS can be valued as meaningful. However, the worldwide HIV/AIDS scenario remains bleak, particularly in terms of behaviours change among men in the visited countries. Further, denial and stigmatisation remain important problems in the daily fight against the epidemic. An intensified response and an increased investment in best practices and innovative approaches are required. The evaluators are of the opinion that the CFAs can take up a leading role and have formulated six recommendations to guide them:
To continue to enhance HIV/AIDS competence within each CFA
To strengthen the capacities and expertise of counterparts, establishing a more in-depth dialogue
To enhance collaboration: between CFAs, between counterparts, between local level, regional and international level.
To continue the implementation of the donor guidelines and adopt local solutions regarding financing insurance costs
To strengthen the comparative advantages of the CFA’s being “generalist “organisations and thus investing in HIV/AIDS mainstreaming. At the same time support AIDS focussed interventions and establishing links between the two type of organisations (AIDS focussed and generalist organisations) in order to reach scale.