2.4. Constraints faced during the execution and limitation of this evaluation assignment
In its implementation the proposed methodology worked well. It was a lot of work for the team to respond to all of the indicators and judgement criteria for the four countries and the five CFAs visited, but the result is good and delivers a lot of evidence for answering the four main evaluation questions.
Some constraints have been noticed in the course of the evaluation exercise:
-
Between 13 October 2005 and 30 January 2006 a lot of work has been done (5 evaluation teams visited 4 countries; 4 country reports, 5 organisational reports and a synthetic document were drafted). In particular, the team underestimated the administrative workload of organising all the missions with the great number of counterparts to be visited in the different countries. In the end we succeeded in organising well all the missions we planned. The visit to Indonesia was cancelled at a late stage and was replaced by a second mission to India. Plan Netherlands preferred that the suggested Plan partners would not be included in this evaluation, as they had no HIV/AIDS programmes. This resulted in an imbalanced cost-effectiveness of the Indonesia mission (too few partners to be visited) and it was therefore replaced by a second mission to India.
-
The team lost some valuable time in switching at the very last moment from a prepared visit to Indonesia to a second mission in the South of India. This brought the number of low prevalence countries in Asia to be visited to only one (India), but we don’t have the feeling that this affects the results of the evaluation, bearing in mind that India on its own is a continent with a lot of differences between the South and the North and as such can represent well the low prevalence regions in Asia.
-
Impact measuring was almost impossible in this kind of short term evaluations. Moreover; the team noticed the poor number of baseline studies or monitoring and evaluation reports in the HIV/AIDS sector, which are absolutely necessary to do anything on impact and change behaviour.
-
HIV/AIDS being a cross cutting issue for most of the CFAs, that brought quite some problems in the financial analyses and the establishment of a sharp project/programme portfolio per CFA. Indeed a lot of financial data were not clear cut as well as the number of projects to be brought or not under the heading of HIV/AIDS. Together with the HIV/AIDS officers of each of the CFAs the team could draw a line and that resulted in the end in good results for using the data in the synthetic document.
-
As a lot of events and initiatives were taken by the CFAs in the course of 2005, the team decided to bring all that into their evaluation (except for the financial data), although the TOR stipulated that the period of the evaluation was limited to the period from 2001 to 2004.
Limitation of this assignment:
-
As already highlighted, this evaluation is mainly an accountability study not aiming to compare the different CFAs. No in depth research has been done on good practices or “best fit models”.
-
The CFA’s themselves were subject of this evaluation, not their counterparts or their international allies, like Oxfam International, Aprodev, Plan International, etc. The evaluators acknowledge that there is close collaboration with these organisations who often take up an important task in the interventions. The collaboration with these partners nor the effectiveness, relevance and efficiency of their interventions has been analysed. The assessment was focussed on the direct inputs of the CFAs.
-
Four of the five CFAs (Novib, Hivos, Cordaid, ICCO) started to collaborate together with the AIDS Fund and created Stop AIDS NOW! This cooperation is institutionalised and operationalised by the creation of a SAN! Secretariat with an own budget and programme. The ToR did not request an evaluation of SAN!. The evaluators have had an interview with the director of SAN! to discuss the cooperation between the CFAs.
Dostları ilə paylaş: |
