Of 29 ethiopia social accountability program (esap2) midterm review (mtr) final report

 
40. Program Impact Evaluation (IE). To complement the MA M&E efforts and enhance potential for 
program impact measurement, the World Bank initiated an impact evaluation that is based on a 
rigorous Randomized Controlled Trial (RCT) at the woreda level. The RCT based IE was designed to 
ensure that the treatment and control groups are statistically comparable, and is envisaged to enable 
an unbiased estimate of the average program impact at the end of ESAP2. The baseline study for the 
IE was approved by the ESAP2 SC on March 28, 2013 and Selam Development Consultants was 
competitively selected to undertake the survey and produce a detailed survey report and contracted in 
June 2013. Baseline data collection began in November 2013. A timeline of two months was initially 
envisioned for baseline data collection, but later extended because the World Bank’s supervision team 
observed some data quality challenges when monitoring and auditing the data collection done.

---

Page 13 of 29 
Based on the World Bank supervision team’s analysis and observations, Selam re-surveyed 4 of the 
study’s 38 woredas in order to produce a final data set which was approved by the World Bank. A 
draft baseline report was submitted by Selam Consultants on April 20, 2014.
41. The 
survey 
collected 
baseline 
information 
on 
the 
extent 
of 
knowledge 
of 
and 
participation/engagement in basic public services delivery; household access to and usage of basic 
services; and their perceptions of basic service delivery in the intervention sectors, especially its 
quality. In addition, information on a wide variety of ‘control’ variables such as level of education, 
poverty profile, and occupational status of the household members, in order to understand how 
different populations interact and are impacted differently by ESAP 2 programming was done. The 
results of the ESAP 2 on initial analysis yielded findings comparable to previous studies of citizen 
engagement in service delivery in Ethiopia, and to other household level data sets in Ethiopia
9
. In 
general, the data suggests significant room for improvement in citizen engagement with services, 
relationships with public officials, and service quality. A detailed comparison of the results of this 
baseline and the results of the ESAP 2 operational baseline survey undertaken by the Management 
Agency is underway. 
42. The TAG held discussions about highlights from the IE findings on May 8, 2014. The full draft report 
from the consulting firm together with a summary report prepared by the Bank team will be shared 
with the TAG for information and feedback during May followed by a presentation at the ESAP2 SC 
meeting. The finally agreed report findings will serve to inform subsequent communications and 
results stories about ESAP2 that the Bank team may plan or be requested to undertake in future. An 
end-line survey is planned at the end of the project
10
. A detailed overview of findings of the World 
Bank implemented IE is provided in annex 4.