Ilo evaluation



Yüklə 2,52 Mb.
səhifə39/41
tarix18.01.2019
ölçüsü2,52 Mb.
#100210
1   ...   33   34   35   36   37   38   39   40   41



Focus Area of Evaluation:

Intervention progress and effectiveness


Is the project on track in delivering its outputs in all countries?

  • Extent to which project activities are implemented according to timeframes

  • Availability of documentary evidence of implementation

  • Tri-partite stakeholders

  • Service Providers

  • Business owners

  • Workers

  • SCORE monthly, biannual & annual reports

  • Interviews (ind. & group)

  • Focus groups

  • Survey

  • Desk-study

  • SCORE Data-base and M&E system

To what extent has the project so far achieved its objectives (incl. the cost recovery plan) and reached its target groups? Do project outcomes contribute to gender equality?

  • Number of institutions that have embedded SCORE training in their service portfolio

  • % of training costs recovered by non-ILO resources so far

  • Number of institutions and corporations that sponsor/pay for SCORE

  • % of Modules delivered with medium or high independence

  • Number of SMEs trained and visited (% of female owned SMEs) and % of SMEs that have formulated and implemented action plans(s) after training

  • Number of male and female managers and employees trained and their perception and appreciation of quality of trainings

  • % SMEs reporting cost saving, reduction absenteeism and employee turnover (% women), reduction of defects, waste, accidents, improved energy efficiency and increased employee satisfaction, both male and female

  • Number of new partner initiatives promoting responsible workplace practices after SCORE training

  • Tri-partite stakeholders

  • Service Providers

  • Business owners

  • Workers

  • SCORE monthly, biannual & annual reports

  • Interviews (ind. & group)

  • Focus groups

  • Survey

  • Desk-study

  • Gender assessment

  • SCORE Data-base and M&E system

Concerning the institutional level, how far has the capacity of partner organizations been built in relation to delivery of the outputs/objectives?

  • Number of representatives of industry associations and training institutions trained (% of women)

  • Number and quality of business plans incl. marketing & communication plans for SCORE training developed by partner organizations

  • Presence of a national centre of excellence for SCORE

  • Number of national expert trainers available per country

  • Existence of Trainer of Trainers curriculum and number of trainers certified in SCORE (% of women)

  • Availability of training materials translated and adapted to national context

  • Number of partners trained to include gender equality concerns and extent to which gender is mainstreamed in training modules

  • Functioning of M&E database


  • Tri-partite stakeholders

  • Service Providers

  • Bi-annual & annual reports

  • Interviews (ind. & group)

  • Focus groups

  • Survey

  • Desk-study

  • Gender assessment

Concerning the enterprise-level, is the training programme effective in leading to the desired impact on enterprises and their workforce? Are the factory improvements lasting? Are SMEs enrolling in more than one Module? If not, why? Has the project targeted the right market and created the right channels for the SMEs to enrol in more than one Module?

  • Number of SMEs trained in Module 1

  • % of SMEs taking up more than one training module

  • Appreciation of the quality, relevance and extent to which trainings address companies’ needs

  • % of SMEs that have formulated and implemented action plans(s) after training

  • % SMEs reporting over a longer period of time cost savings, reduction absenteeism and employee turnover (% women), reduction of defects, waste, accidents, improved energy efficiency and increased employee satisfaction (male and female)

  • Argumentation of SMEs that did not follow more than one module

  • Service Providers

  • Business owners

  • Workers

  • SCORE monthly, biannual & annual reports

  • Interviews (ind. & group)

  • Focus groups

  • Survey

  • Desk-study

  • Company walk-through

  • Gender assessment

  • SCORE Data-base and M&E system

What obstacles did the project encounter in project implementation? What corrective action does the project need to take to achieve its objectives?

  • This question does not refer to indicators and will be answered in the chapters of conclusions and recommendations.

  • Tri-partite stakeholders

  • Service Providers

  • Business owners

  • Workers

  • SCORE monthly, biannual & annual reports

  • Interviews (ind. & group)

  • Focus groups

  • Desk-study




Efficiency of resource use


Does the project make efficient use of its financial and human resources?

  • Comparison of total staff time dedicated to SCORE at country and global level

  • Turnover over staff

  • Amount and frequency of over- and/or under-expenditure in project / per country

  • SCORE and implementing org staff

  • Subcontracted service providers

  • SCORE & implementing org staff

  • Subcontracted service providers

  • SCORE monthly, biannual & annual reports

  • Interviews

  • Desk Study

Is the implementation strategy cost-effective?

  • % Administration costs

  • Comparison costs per module/trainer/trainee per country

  • SCORE and implementing org staff

  • Subcontracted service providers

  • SCORE monthly, biannual & annual reports

  • Interviews

  • Desk Study

Is the distribution of resources between staff and activities optimal?



  • % Administration costs

  • Distribution budget global and country level

  • SCORE monthly, biannual & annual reports

  • Interviews

  • Desk Study

Were the intervention resources used in an efficient way to address gender equality in the implementation?

  • % of budget invested in specific gender related activities

  • Project expenditures per female beneficiary compared with male beneficiaries

  • SCORE and implementing org staff

  • Subcontracted service providers

  • SCORE monthly, biannual & annual reports

  • Interviews

  • Desk Study

  • Gender assessment

Focus Area of Evaluation:

Effectiveness of management arrangements


Are time frames and work plans respected? How are contingencies dealt with? To what extent corrective action is taken when required?

  • Amount and magnitude of delays compared per country

  • Amount and kind of external factors influencing implementation. % of cases in which proven mitigating actions were taken

  • SCORE and implementing org staff

  • Subcontracted service providers

  • SCORE monthly, biannual & annual reports

  • Interviews

  • Survey

  • Desk Study




Is the management structure effective?

  • Appreciation of ILO staff, implementing partners and sub-contracted service providers on the effectiveness of management structure

  • SCORE and implementing org staff

  • Subcontracted service providers

  • Interviews




Has the project staff sought and received adequate support from the global component, relevant ILO units and offices, including gender expertise when needed? Does the country project staff have sufficient authority/delegation in executing and managing the project (e.g. technically, financial and admin. management)?

  • Appreciation of ILO staff, implementing partners and sub-contracted service providers on the degree of autonomy and effective and efficient decentralised operations at the country level

  • Appreciation of ILO staff, implementing partners and sub-contracted service providers on the timeliness and quality of central service delivery, including specific gender expertise

  • SCORE and implementing org staff

  • Subcontracted service providers

  • Interviews




Should the project develop regional hubs for support?

  • This question does not refer to indicators and will be answered in the chapters of conclusions and recommendations.







Are donor relations, administration of processes, and project monitoring by donor Headquarters and country staff effective?

  • Appreciation of ILO staff of administrative dialogue and cooperation and specific requirements of SECO and NORAD

  • Frequency and kind of contact/dialogue between SCORE and national representative offices of SECO and NORAD at country level

  • SECO & NORAD

  • ILO central and decentralised staff

  • Interviews

Are the Global and National Tripartite Advisory Committees functioning and what value do they add?

  • Frequency of meetings and attendance by the tri-partite members

  • Appreciation of dialogue and outcomes of N and/or G TAG meetings by tri-partite partners.

  • SECO & NORAD

  • ILO central and decentralised staff

  • ITUC and IOE (and their national members)

  • Minutes G&N TAC

  • Interviews

  • Desk Study

Is project systematically and appropriately monitoring, docu-menting & communicating results, including on gender, at the country and global level? Is M&E system practical, useful & cost effective for project management? Does manage-ment use information generated by M&E system to assess progress against the objectives (including gender-related results) and take necessary adaptive measures when required? Should indicators be revised?

  • DCED qualification of SCORE M&E systems

  • Appreciation of ILO staff and implementing partners of quality and usefulness of information

  • Number and kind of revisions and adaptations that were made based on interpretation of M&E data, including specific gender actions

  • Amount of time and effort needed to generate M&E data and to retrieve M&E data from on-line database and SMART sheets

  • % indicators that is not measured and/or are systematically not achieved

  • DCED audit report

  • M&E systems

  • Monthly, bi-annual and annual reports

  • SCORE staff and implementing organizations staff

  • Interviews

  • Desk Study

  • M&E system assessment

  • Gender assessment

How effective is the project in sharing good practices between country components and communicating success stories and disseminating knowledge internally and externally (including gender-related results and knowledge)?

  • Number and kind of information exchange and meetings at the ILO, regional, global level

  • Number and kind of information exchange and meetings with external stakeholders at regional, global level

  • Incorporation or replication of SCORE lessons in other countries and or by other stakeholders (gender lessons specified)

  • SCORE and implementing organizations staff

  • External stakeholders

  • Annual Reports

  • Interviews

  • Desk Study

Focus Area of Evaluation:

Impact orientation and sustainability of the intervention


How effectively has the project built national ownership and capacity of people and institutions? Are national partners (employer organizations and industry associations) willing and able to continue the project after funding ends?

  • Appreciation of SME owners of SCORE programme

  • Self assessment of capacity of training service providing organizations

  • Degree of willingness to pay for training services

  • Appreciation of employer organizations and business associations of the programme

  • Willingness to continue the project after ILO moves out




  • SME owners

  • Training providers

  • Tri-partite partners

  • SCORE and implementing organizations staff




  • Interviews

  • Focus group meetings

  • Survey

Are there business models applied in the different countries that seem more promising to reaching financial sustainability?

  • Degree of financial sustainability of other enterprise training and capacity development programmes (as far as information can be found on this)

  • Number and kind of proposals of business community for other business models for enterprise training

  • SME owners

  • Training providers

  • Tri-partite partners

  • SCORE and implementing organizations staff

  • Interviews

  • Focus group meetings

Are the gender-related outcomes likely to be sustainable?

  • Appreciation and assessment of employers and workers on quality and sustainability of gender outcome

  • Evaluators’ gender assessment of outcomes

  • SME owners

  • Training providers

  • Tri-partite partners

  • SCORE and implementing organizations staff

  • Interviews

  • Focus group meetings

  • Gender Assessment

Has the project reached sufficient scale and depth to justify the investment?

  • Appreciation of tri-partite stakeholders on scope and scale of SCORE project

  • Historical growth and outreach of SCORE among service providers, trainers and companies

  • Evaluator’s assessment

  • SME owners

  • Training providers

  • Tri-partite partners

  • SCORE and implementing organizations staff

  • Interviews

  • Focus group meetings




Has the project found the right balance between scale and depth and the trade-off between them?

  • Appreciation of tri-partite stakeholders on scope and scale of SCORE project

  • Evaluator’s assessment on Balance

  • SME owners

  • Training providers

  • Tri-partite partners

  • SCORE and implementing organizations staff

  • Interviews

  • Focus group meetings

Are the approach and its results likely to be up-scaled or replicated?

  • This question does not refer to indicators and will be answered in the chapters of conclusions and recommendations







Is the project a cost-effective way to improve productivity and working conditions in SMEs?

  • This question does not refer to indicators and will be answered in the chapters of conclusions and recommendations












ANNEX 13
Terms of Reference






SCORE – Sustaining Competitive and Responsible Enterprises

Independent Mid-Term Evaluation

Title:

Sustaining Competitive and Responsible Enterprises (SCORE)

TC Code:

Global SCORE (GLO/13/55/MUL)

SCORE China (CPR/13/50/MUL)

SCORE India (IND/13/50/MUL)

SCORE Indonesia (INS/13/51/MUL)

SCORE South Africa (SAF/13/50/MUL)

SCORE Ghana (GHA/13/50/MUL)

SCORE Colombia (COL/13/50/MUL)

SCORE Viet Nam (VIE/13/51/MUL)



Backstopping unit

SME Unit

Type of evaluation

Independent Mid-Term Evaluation

Evaluation Manager

Annie van Klaveren

List of Acronyms

AA

Administrative assistant

BDS

Business development services

CTA

Chief technical advisor

DCED

Donor Committee for Enterprise Development

DWCP

Decent Work Country Programme

EM

Evaluation manager

EVAL

Evaluation office

HEADQUARTERS

Headquarters

ILO

International Labour Organization

IOE

International Organization of Employers

ITUC

International Trade Union Confederation

KPIs

Key performance indicators

M&E

Monitoring and evaluation

NORAD

Norwegian Agency for Development Cooperation

NPC

National project coordinator

NPO

National project officer

P&B

Programme and budget

PPPs

Public private partnerships

SCORE

Sustaining Competitive and Responsible Enterprises

SECO

Swiss State Secretariat for Economic Affairs

SMEs

Small and Medium Enterprises

TA

Technical assistant

TAC

Tripartite Advisory Committee

TC

Technical committee

TO

Technical officer

UN

United Nations

UNDAF

United Nations Development Assistance Framework

UNEG

United Nations Evaluation Group

USD

United States dollar


Yüklə 2,52 Mb.

Dostları ilə paylaş:
1   ...   33   34   35   36   37   38   39   40   41




Verilənlər bazası müəlliflik hüququ ilə müdafiə olunur ©muhaz.org 2024
rəhbərliyinə müraciət

gir | qeydiyyatdan keç
    Ana səhifə


yükləyin