Ilo evaluation

Management Arrangements and tentative Time Frame
Management Arrangements

The final independent evaluation will be conducted by an independent evaluator. The evaluator can constitute his/her team as he/she sees fit (hiring additional staff from the local countries for example). All members of the evaluation team (including the additional staff) shall thus fall under his/her supervision and responsibility.

The independent evaluator is responsible for conducting the evaluation according to the terms of reference (ToR). 
On the ILO’s side, the evaluation will be supervised by the Evaluation Manager. The Evaluation Manager will:

Ensure meeting schedules are set up;

• 
  Assist in the implementation of the evaluation methodology, as appropriate (i.e., participate in interviews, observe committee meetings) and in such a way as to minimize bias in evaluation findings;
  
• 
  Review and provide comments on the evaluation report;
  
• 
  Ensure that the evaluation is conducted in accordance with these terms of reference, for the preparation of the draft report of the evaluation, discussing it with the evaluator, the beneficiaries and the stakeholders;
  
• 
  Liaise with SCORE project staff wherever their engagement is needed to fulfil the requirements above.
  

Application requirements

Selection of the contractor will be done by the ILO based on their technical and commercial proposals. Proposals to undertake any work under these ToRs will be submitted in English and must contain the following information and documents:

1. Technical Proposal

• 
  A short summary of profile and capacity of the Contractor to conduct an evaluation of a private sector development project, including a record of relevant work executed in the past five years;
  
• 
  A proposal on how the contractor intends to complete the work described in the ToRs;
  
• 
  The CV(s) of the lead evaluator and other team members that will undertake the work;
  
• 
  A timeline with proposed dates for contract start and end dates and tentative dates for country visits (taking into account visa processing times).
  

2. Commercial Proposal

A proposal setting out the cost for the evaluation including a daily fee (or daily fees in case several team members will be involved in the evaluation), nr. of work days per staff, and tentative travel costs per mission. The Financial Offer form is enclosed as Annex II-F.
Requirements of the evaluating company and the evaluator(s)
1. The company

• 
  Evaluation should be part of the company’s services offer. The company should have evaluated at least one private sector development project by bilateral or international organizations in the past five years.
  
• 
  The company should demonstrate gender balance and diversity in their staff and commitment to the principles of gender equality and non-discrimination.
  
• 
  Familiarity with the ILO mandate, its tripartite structure and international labour standards is a plus.
  
• 
  Operations or presence of local offices in countries which are being evaluated is a plus.
  

2. Lead evaluator

• 
  Knowledge, skills, and experience (at least ten years) in the area of M&E and evaluation.
  
• 
  Knowledge and experience (at least six years) of enterprise operations and the provision of business development services (BDS) to SMEs. Experience with the Donor Committee for Enterprise Development (DCED) standard for results measurement is a plus.
  
• 
  Experience as a project manager/team leader (at least five years).
  
• 
  Relevant country experience in one or more of the project countries under review is an advantage.
  
• 
  Excellent written and oral communication skills in English (level C2). Proficiency in Spanish is required unless it is provided by another team member of the evaluation team.
  

3. Evaluation team members

• 
  The evaluation team proposed by the company should be composed of sufficient number of well-trained project team members to fulfil assignment obligations. The following requirements apply to any team member leading a field visit to a project component:
  
• 
  Team composition brings experiences on relevant knowledge areas (M&E, evaluation, Donor Committee for Enterprise Development (DCED) results measurement standard, provision of business development services (BDS), labour standards compliance, CSR) and geographic and language expertise.
  
• 
  Spanish is a must if not covered by the lead evaluator.
  
• 
  All team members proposed to conduct field missions have at least seven years experience.
  
• 
  Relevant country experience in one or more of the project countries under review is an advantage.
  
• 
  Excellent written and oral communication skills in English (level C2). Proficiency in Spanish is required unless it is provided by the lead evaluator or another team member of the evaluation team.
  
• 
  Gender balance in the team composition is expected.
  

Replacement of evaluation team members

The Evaluation Team assigned by the Contractor to perform the services under this Contract, which is considered essential for the performance of those services, shall be composed of the Personnel indicated in the Technical and Financial Proposal of the Contractor. Accordingly, in addition to sub-paragraph 4.5.3 of the Terms and Conditions applicable to ILO Contracts for Services (Annex IV):

• 
  if any of the Contractor’s Personnel part of the Team is removed or for any reason is no longer available to perform the services then the replacement Personnel shall be of equal or better knowledge, experience and ability to perform the services;
  
• 
  prior to replacing any Personnel part of the Team, the Contractor shall notify the ILO reasonably in advance and shall submit detailed justifications together with the resume of the proposed replacement Personnel to permit evaluation by the ILO of the impact which such Personnel replacement would have on the work plan;
  
• 
  no Personnel replacement of the Team shall be made by the Contractor without the prior written consent of the ILO, which approval shall not be unreasonably withheld; and
  
• 
  The Contractor will not charge the ILO for any additional costs in supplying any replacement Personnel.
  

The introduction of replacement of any Personnel part of the Team may constitute considerable losses for the ILO. Therefore, the Contractor’s Personnel are expected to perform the services until the completion of the assigned tasks and deliverables.

In the event of demonstrable poor performance or misconduct of the Personnel part of the Team, if the ILO so decides, the Contractor shall provide an appropriate replacement for such Personnel. The Contractor will provide suitable replacement personnel within 15 work days. The provisions of subparagraphs 9.1 and 9.2 will apply.


Compliance with UN norms and standards for evaluation
This evaluation will comply with UN norms and standards for evaluation and ensure that ethical safeguards concerning the independence of the evaluation will be followed. Please refer to the UNEG ethical guidelines: http://www.unevaluation.org/ethicalguidelines

To ensure compliance with ILO/UN rules safeguarding the independence of the evaluation, the contractor will not be eligible for technical work on the project for the next 12 months and cannot be the evaluator of the final project evaluation.

1^ ILO (2009): Sustainable Enterprise Programme: Promoting Job Creation through the Development of Sustainable Enterprises – Strategic framework. The programme is a response to the Conclusion concerning the promotion of sustainable enterprises formulated by the ILO constituents at the International Labour Conference in 2007.

2^ The project in Bolivia is funded separately by the Norwegian Ministry of Foreign Affairs and ILO RBSA funds.

3^ ILO (2013): SCORE Phase I Final Report.

4^ The budget for Phase I includes 1.3 million US dollars of NORAD funds, which were originally allocated to Phase II. NORAD advanced these funds to ILO as the signing of Phase II was delayed, and these funds were thus formally administered under Phase I.

5^ ILO (2013): SCORE Phase II Project Document.

6^ ILO (2013): SCORE India: final internal evaluation.

7^ ILO, Programme and Budget for the Biennium 2012-13.

8^ Vietnam, however, was building on the Factory Improvement Programme (FIP) which was implemented by ILO and VCCI until 2009.

9^ For India, the number is not entirely correct since some people who attended training as observers were by accident added as trainer candidates. This still needs to be corrected in the M&E database.

10^ The M&E database shows slight differences in the records of individual trainers that are sometimes higher than in the summary country reports. For the analysis here the evaluators have used the country report data.

11^ This evaluation is a Mid-Term Evaluation and no final evaluation with an impact analysis. Therefore the company visits were not conducted to make a comprehensive analysis of effects and impact of SCORE at the SME level. Visits to SME’s were conducted only to companies that still were actively implementing SCORE and therefore the evaluators have looked to good practices of SCORE and the insights obtained here refer to potential effects of SCORE in SME’s that have taken ownership of it.

12^ Assessments during walk-throughs were done on a 3 point scale ranging from 1 (no or weak effects) to 2 (considerable effects) and 3 (major changes observed).

13^ SCORE Phase II Prodoc (p. 12).

14^ The budget for SCORE Phase I was 9.7 million US dollar while the budget for Phase II is 18.1 million US dollar. Assuming that at the end of 2015 at mid-term of Phase II half of the budget will be spent, the total expenditures to data are close to 19 million US dollars. This amount divided by the number of beneficiaries is the basis of calculation of costs per company.

15^ In Colombia there was no specific NTAC for SCORE as different ILO programmes were combined in one overall NTAC. This was recently changed to enable trade unions to become more directly involved in steering SCORE.

16^ A first study was done in collaboration with ACTRAV: Worker participation in SMEs: perspectives from the aluminium industry in Ghana.

17^ Outcome mapping and outcome harvesting as a methodology do not start with predetermined outcomes and measure progress towards them, but rather collect evidence of what has been achieved in the programme and work backwards to determine whether and how the programme or interventions contributed to the change. For more information and examples of outcome mapping and outcome harvesting see the recent Learning Paper of Saferworld: Doing Things Differently.

18^ Although the 22,300 USD of average savings achieved in China might not be fully reliable because of not always reliable M&E data on this (and other) KPI’s. However, it is likely that due to size and characteristics of the often considerably sized enterprises are much larger than in other SCORE countries.

19^ www.worldbank.org/en/country/ghana/overview

20^ www.data.un.org

21^ IOL Business News, 13 November 2015

22^ www.worldbank.org/en/country/ghana/overview

23^ Isaac Quaye et al (2015): Bridging the SME Financial Gap in Ghana

24^ SCORE Ghana Project Strategy Document, 2010

25^ SECO: Indonesia Country Strategy 2013-2016

26^ World Bank, Indonesia Economic Quarterly, July 2015

27^ ILO SCORE Project Strategy document for Phase II

28^ Confirmed in Market Assessment for SCORE, Spenceley at all 2013 – 97% of employers are SMME/ 23.000 companies in hospitality sector

29^ Market assessment SCORE – in annex challenges are presented

30^ Also expressed in Market Assessment SCORE

31^ Kwazulu Natal, Free state, Eastern Cape, Department of Economic Development, Environmental Affairs and Tourism

32^ The previous projects in both South Africa and China have been subjected to independent final evaluations. The evaluation reports will be made available to the evaluator.

33^ Project activities in Peru started with funding from the Canadian Ministry of Labour but are now funded under the global SCORE project umbrella.

34^ Sectors are chosen through a transparent sector selection study and process in consultation with ILO field offices, ILO constituents and the donors.

35^ Key questions under each evaluation criteria have been designed to help address the extent to which the mainstreaming of gender equality has been integrated into the implementation of the intervention, the effectiveness and efficiency in mainstreaming gender equality, the outcomes delivered in terms of gender equality, and an estimation of the impact of the policies implemented on the equality of women and men, when appropriate.