Peru Second Phase of Health Reform Program Implementation Completion and Results Report

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation

The Overall Relevance Rating is: Substantial.

34. 
    Relevance of Objectives: Rating High. The project was fully aligned with the CPS for FY12-16 for Peru (February 1, 2012; p.21) and with the priorities of MINSA for an investment program with a strong focus on reducing maternal mortality, infant mortality and undernutrition, and an interest in improving equity in access to services. Today, reduction of maternal mortality and especially of infant mortality remain important, while non-communicable diseases are increasingly capturing attention and funding due to increased burden. Malnutrition and anemia still remain very high on the Government agenda given that progress has been fluctuating, not only in the project areas, but more generally across the country. In addition, poverty and inequalities, especially in the project areas, are still far from being resolved. The World Bank is currently undertaking the first Systematic Country Diagnostics for Peru (the review meeting will be held before the end of FY16) and will start working on the Country Partnership Framework in the first quarter of FY17. Addressing inequalities, support to effective decentralization, and improved nutrition, all of which were at the core of PARSALUD, still remain highly relevant.
    

35. 
    Relevance of Design: Rating Substantial. Given that this project supported phase II of the program and all triggers were met at the end of phase I, the APL design was maintained. MINSA gained experience in managing APL fiduciary rules and believed these would help reinforce accountability, expressing preference for this lending instrument.
    

36. 
    The project design appropriately aimed to address both demand and supply side factors to improve maternal and child health outcomes in the nine regions, as well as to increase the capacity of the government at the national and local level. It had a strong intercultural footprint, which was very appropriate given the areas of focus under the program. It was also designed in parallel and to build synergies with another Bank-financed project aimed at improving nutrition outcomes by using conditional-cash transfers, the Juntos Results for Nutrition Project (P117310), approved in 2011.
    

37. 
    As mentioned above, the RF was aligned with the project components and reflected the different parts of the PDO. At approval, the design of the RF still reflected baseline data of 2005, which could have been updated with more recent data, either at that stage or at the first restructuring in 2011. Data for some IOIs was not available and five indicators were dropped at the 2014 restructuring. More effort could have gone to incorporate in the project’s RF some IOIs included in the program’s RF to better assess progress with Component 1 and 3; to compensate for these weaknesses, several IOIs were included at the time of the ICR.
    

3.2 Achievement of Project Development Objectives

The Efficacy Rating is: Substantial. The overall efficacy rating is the result of the assessment of achievement of the three parts of the PDO, all of which are rated substantial.
Assessment of achievement of the program’s objectives

38. 
    The project objective was to contribute to the overall GOP’s effort to reduce maternal and child mortality and improve chronic malnutrition in children ― and, as explained below, the project did contribute to this. A comprehensive impact evaluation would be needed to demonstrate the attribution of the achievement of the program objectives and of the PDO-level indicators the project. However, the case for project contribution to the achievement of the PDO, is supported by progress on the intermediate outcome indicators that are more directly attributable to the project.
    
39. 
    In terms of the overall results of the program, between 2009 and 2014, Peru was able to reduce IMR from 42 to 17 against a target of 25, and chronic malnutrition of children under 5 years of age from 38.2% to 23.7%, against a target of 30.2% (last PARSALUD progress report). MMR per se was not monitored by the program due to uncertainty about the reliability of data; however, the indicator on institutional deliveries was used as a proxy. As seen below, this indicator moved from 62.2% to 74.2% against a target of 78%, between 2009 and 2014. The total number of maternal deaths was also reduced from 186 to 159 in the project areas, against a national average of 481 and 411 between 2009 and 2014. Hence, overall, PARSALUD was successful in reducing MMR and IMR and chronic malnutrition in children under 5 years of age. More detailed program results are found in Annex 3.
    

Assessment of achievement of the project’s objectives

40. 
    In terms of the three specific project objectives, the ICR assesses the original indicators as well as additional indicators added at the time of the ICR to introduce additional evidence in support of the results chain to the PDO. All indicators were related to the three parts of the PDO statement, as follows:
    

1. 
   Improvement of family care practices for women (during pregnancy, delivery and breast-feeding), and children under the age of three:
   

KPIs# 1, 2, 3, 4, 6; IOIs # 1, 11

2. 
   Strengthening of health services networks with capacity to solve obstetric, neonatal and infant emergencies and to provide comprehensive health services to women (during pregnancy, delivery and breast-feeding) and children under the age of three:
   

KPIs# 1, 2, 4, 5, 6; IOIs # 2, 3, 4, 5, 6 and 12 (added)

3. 
   Supporting MINSA's governance functions of regulation, quality, efficiency and equity for improving the new health delivery model of maternal and child health care in a decentralized environment
   

IOIs # 7, 8, 9, 10; and 13, 14, 15 (added).

41. 
    IOIs #2 and 5 (PDO 2), 7, 9 and 10 (PDO 3) were dropped at the 2014 restructuring. At the time of the ICR data was only available to support IOIs #2 and 5, which were therefore reintroduced. Additional qualitative data was collected to support the analysis of PDO 3.
    

42. 
    When assessed against the original baseline, all the indicators have surpassed, achieved or partially achieved the targets at the end of the project, which supports achievement of all parts of the PDO (Table 1). Also, notably, the final percentage of achievement does not reflect that roughly half of all indicators surpassed their targets. Therefore, the actual level of achievement is far greater than the table suggests.
    

Table 1: Achievement of PDO (targets against original 2005 baseline)

	Program level	PDO level	Intermediate	PDO 1*	PDO 2*	PDO 3
Target surpassed	2	3	6	4	4	3
Target achieved or substantially achieved (>=85% met)		2	4	1	5	1
Target partially achieved (65%-84% met)		1	2	2	2	0
Target not achieved (<65% met)		0	0	0	0	0
Unknown		0	3	0	0	3
Total	2	6	15	7	11	7
% surpassed and achieved	100%	83%	83%	71%	82%	100%

* Note: The table double-counts some indicators to assess PDO 1 and 2.

43. 
    Given the fact that the baseline for this project which was approved in 2009 actually used 2005 data (for reasons explained above), the ICR team reviewed achievement of project indicators against the 2009 baseline (see Annex 10). Notably, many of the indicators already registered improvements in 2009 compared with the baseline values of 2005. This overall trend continued during project implementation, with all but one indicator (prevalence of anemia in pregnant women) following the positive trajectory. This supports the positive contribution of the project to the achievement of the PDO. In addition, many of the indicators for the project regions showed performance at least in line with the national average (see Annex 3), a very positive result considering the geographical and socio-economic conditions of the project regions.
    

44. 
    The assessment below is based on the official 2005 baseline data, and when appropriate, also uses revised targets for IOIs.
    

PDO 1: improvement of family care practices for women (during pregnancy, delivery and breast-feeding), and children under the age of three. Rating: Substantial.
KPI #1: Increase the proportion of institutional deliveries in rural areas of the nine selected Regions from 44% (2005) to 78% (2014) – Achieved

KPI #2: Reduce the prevalence of anemia among children under the age of 3 in the nine regions from 69.5% (2005) to 60% (2014) – Surpassed

KPI #3: Increase from 64% to 80% the share of children in the nine selected regions who are exclusively breastfed until 6 months of age – Surpassed

KPI #4: Reduce the prevalence of anemia among pregnant women in the nine Regions from 41.5% (2005) to 35% (2014) – Partially Achieved

KPI #6: Increase in the proportion of pregnant women of the nine regions with at least 1 prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2014) – Surpassed

IOI #1: Percentage of SIS affiliated children who received growth and development controls (CRED) according to their age – Partially Achieved

IOI #11: Number of health personnel and community health workers trained within the behavior change campaign (EPPES) – Surpassed

45. 
    The project contributed to the increase of breastfeeding practices, one of the most important indicators for Component 1. Exclusive breastfeeding has been low in Peru, although in the project areas it was higher than the national average in 2009 (82.2% vs. 68.5%). The project greatly contributed to the uptake of breastfeeding practices in the nine regions, since the share of exclusive breastfed children in project areas rose to 87%, while the national average remained flat at 68.4%. This was in line with other GOP programs such as Juntos, although other programs aimed at providing food and formula supplements to poor families; therefore, achievements under the project seem to be remarkable.
    

46. 
    Progress on anemia in the nine regions has been slow, reflecting national trends. Anemia has traditionally been a problem in Peru, often associated with lack of knowledge. Anemia in children under 3 has been stable between 2009 and 2014 in the project areas, while it has slightly declined as a national average (50.4 and 46.8). Anemia in pregnant women declined between 2005 and 2014 but it did not follow a steady trajectory. The prevalence of anemia, higher in the nine regions compared to national average, was relatively stable at the beginning of the project, at around 30%. However, after dropping to 24.3% in 2013, well below the national average of 28%, it started rising again to levels higher than the national average (36.4% vs 32.5%). In the project regions, the reasons for this trend seems to be related more to demand-side factors than supply-side factors. The availability of iron supplements at facility level has increased over time and the proportion of women who receives those supplements has also increased. According to PARSALUD staff, women admit that they are not taking iron supplements due to their bad taste, color and smell. New supplements have recently been purchased to overcome this issue; their effectiveness is yet to be assessed once the data for 2015 becomes available. The slow progress on the prevalence of anemia in women in Peru requires additional efforts to ensure stable and sustainable improvements.
    

47. 
    The achievement of PDO 1 is supported by successful progress on a number of IOIs. All facilities that needed to be provided with audiovisual equipment to support the behavior change campaigns were in fact provided one (1,423). The project trained more a total of 1,178 health personnel and community health workers within the communication and behavior change campaign (400 more than planned). According to SIS data, the biggest increase in the number of services provided to SIS-enrolled beneficiaries between 2011 and 2015 was for primary care services, which demonstrates greater accessibility of basic services for the poor, with a sharp increase in prevention as opposed to curative services. Finally, the campaign promoting identity and insurance rights (Derecho a la Identidad y Aseguramiento, DIA) was conducted twice in every region for 3 months each time, for a total of 6 months of campaing in every region ― 3 months less than originally planned.
    

48. 
    PDO 2: strengthening of health services networks with capacity to solve obstetric, neonatal and infant emergencies and to provide comprehensive health services to women (during pregnancy, delivery and breast-feeding) and children under the age of three. Rating: Substantial.
    

KPI #1: Increase the proportion of institutional deliveries in rural areas of the nine selected Regions from 44% (2005) to 78% (2014) –Achieved

KPI #2: Reduce the prevalence of anemia among children under age in the nine regions from 69.5% (2005) to 60% (2014) – Surpassed

KPI #4: Reduce the prevalence of anemia among pregnant women in the nine Regions from 41.5% (2005) to 35% (2014) – Partially Achieved

KPI #5: Reduce the hospital lethality rate among neonates in the nine selected Regions from 9.5% (2005) to 5% (2014) – Achieved

KPI #6: Increase in the proportion of pregnant women of the nine regions with at least 1 prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2014) – Surpassed

IOI #2: Percentage of health facilities with improvement in infrastructure (minor construction and/or equipment) -- Achieved

IOI #3: Percentage of SIS affiliated rural pregnant women with laboratory tests on hemoglobin, urine and syphilis – Surpassed

IOI #4: Percentage of pregnant women under SIS that receive iron and folic acid supplements – Partially Achieved

IOI #5: Percentage of women satisfied with the services in selected facilities by confidence index – Surpassed

IOI #6: Percentage of cesareans in SIS affiliated pregnant rural women – Achieved

IOI #12: Number of health facilities improved – Achieved

49. 
    Health infrastructure improved in the nine regions. The project supported the construction and/or renovation of 69⁵ out of the planned 73 hospitals and basic health centers that offered obstetric and neonatal services; the remaining 4 are being completed in 2016, financed by GOP. It provided all of the 104 pre-identified health centers with medical equipment for maternal and child care and installed IT systems in 55 centers to support the implementation of the e-Health plans (planned 54).
    

50. 
    The project delivered capacity building programs for different groups of health professionals using a culturally-sensitive approach. The project contributed to the training of 956 health workers (nearly 300 more than planned) on maternal and child care and specifically of 282 doctors and midwives on the vertical delivery (Parto Vertical) in 7 health facilities located in 4 regions within PARSALUD. Together with improved infrastructure and training, cultural adaptation of health facilities for the provision of vertical deliveries and support to Maternal Waiting Homes (Casas Maternas) were key in improving rates of institutional deliveries among indigenous populations. This includes the creation, dissemination, and systematization of knowledge around vertical delivery practices among health care providers and the institutionalization of this method in both regions: in the Amazon and Cusco, 54% and 33% of all deliveries were vertical, respectively, in 2012. The Casas Maternas, communal space managed, built, and maintained by communities and local governments, helped address the physical and cultural barriers faced by indigenous women residing far from health centers. There are 475 operational Casas Maternas throughout the country, with Cusco, Puno, Huancavelica and Apurimac the most important areas of reference.
    

51. 
    The project strengthened the networks of services and improved quality of care and access to emergency services. In order to ensure a more effective network of services, all 9 regions elaborated a plan for the improvement of the referral and counter-referral system. The project also trained key health personnel in management of human and financial resources (1,336 actual vs. 1,143 planned).
    

52. 
    On quality of care, the project funded a specific training and implementation program to reduce maternal pre- and post-partum bleeding through the introduction of a supervised medical protocol based on the use of specific evidence-based interventions and medicines, which has been shown to reduce post-partum hemorrhage. According to a survey done in 2012, the reported satisfaction among women who used the services was 75%.
    

53. 
    The rates of C-sections in women affiliated with SIS increased in line with the intention of ensuring that women in need were actually able to access this service. While the national average has increased to 12%, in the PARSALUD region it seems to have been stable since 2012. This seems to suggest that while excessive use of those services might be starting in Peru, as it is already common practice across Latin America, in the project area those services are provided only when required.
    

54. 
    PDO 3: supporting MINSA's governance functions of regulation, quality, efficiency and equity for improving the new health delivery model of maternal and child health care in a decentralized environment. Rating: Substantial.
    

KPI #8: Percentage of SIS affiliated households that make out-of-pocket expenditures in medicines – Partially Achieved

KPI #13: Norms and regulations to improve efficiency and equity of the health delivery system prepared – Surpassed

KPI #14: Clinical pathways and corresponding financing systems designed – Surpassed

KPI #15: Periodic evaluations of the performance of the health networks – Surpassed

55. 
    The achievement of the IOIs suggests that progress towards PDO 3 was significant. PARSALUD II developed 32 proposals for norms and legal acts to improve efficiency and equity of the health system for approval by MINSA, when they had only planned for 27. The design of the clinical pathways and corresponding financing system was also finalized by the project (16 actual vs 11 planned). The capacity of analyzing performance of the health system also improved through the completion of regular performance reports (19 produced vs. 17 planned) – although the majority of those were released in 2015, rather than being released constantly throughout the project and periodically updated.
    

56. 
    PARSALUD II supported MINSA in reformulating its regulatory framework to ensure better quality of the health delivery system, in particular on purchasing, prescribing, and monitoring the quality of pharmaceuticals. Peru’s consolidation of a purchasing system for essential medicines lowered their costs for the country. In addition, a comprehensive training system in essential drug prescriptions was implemented in the project’s health facilities, and support was provided for improving the regulatory framework for quality control using SUNASA (Superintendencia Nacional de Salud).
    

57. 
    The project strengthened the regional management capabilities on MCH through specialized training in neonatal and obstetric competencies for health professionals and managerial capacities for the Health Regional Directorates (DIRESA). Budget executions for functions related to maternal and child care and especially nutrition improved in the nine regions following capacity building interventions between 2012 and 2014 (e.g. Cusco moved from 88.7% in 2012 to 97% budget execution in 2014 on maternal and child health, while Amazonas moved from 77.5% in 2012 to 91.6% in 2014 on nutrition)
    

58. 
    Additional qualitative information supports significant progress on PDO 3. This information was collected during the ICR mission and triangulated with findings by PARSALUD evaluations derived from focus group discussions and individual interviews conducted with informants at MINSA, PARSALUD team, regional administrators, health professionals, and beneficiaries. The technical assistance provided by PARSALUD supported a cultural change within the regional administrations and health professionals towards an approach that recognizes the human right of pregnant women and children to be treated fairly and with dignity, which has been appreciated by the beneficiaries. The project assisted the decentralization process by providing targeted assistance to the regions that needed more specific support with the regulation function. Topics of focus varied depending on the needs of each region, ranging from human resources to supply chain management. Progress were made on enhancing transparency, with a portal and website created for PARSALUD. In addition, central planning for civil works was strengthened as delays in starting and concluding civil works at the beginning of the project reduced over time. Overall, this evidence suggests that PARSALUD promoted significant progress in improving the government capacity on regulation, quality, equity, and efficiency.
    

3.3 Efficiency

The Efficiency Rating is: Modest

59. 
    Rather than carrying out a separate economic analysis, the PAD referred to the one done in the FS. The ICR could not update the analysis done in the FS; instead, a more qualitative assessment of project efficiency and cost-effectiveness was undertaken (see Annex 3).
    

60. 
    Cost-effectiveness was overall substantial, especially considering that the Bank loan leveraged a much more significant amount of domestic resources; on balance, in consideration of the delays and the disbursement level, efficiency is considered modest.
    

Allocative efficiency

61. 
    Maternal, newborn and child health (MNCH) interventions are recognized by the literature as highly cost-effective investments. Particularly effective interventions in MNCH packages would be management of labor and delivery, care of preterm births, and treatment of serious infectious diseases and acute malnutrition ⁶ ― all key focus of PARSALUD. Such MNCH interventions, targeted to those most in need, can prevent maternal and infant deaths and reduce the healthy years of life lost due to disability, thereby benefiting the health sector and society as a whole ^7,8,9.
    

62. 
    Lack of infrastructure and professional attention, under registration of beneficiaries on the public health insurance (SIS) and poor managerial and logistic capacity on local health units were identified as barriers to health service availability for women and children in rural areas. Therefore, closing the gap in health care access required targeting resources on the identified barriers and on vulnerable rural populations. The return on infrastructure investments have clear long-term benefits, which facilitate improvements in the provision of quality services. In addition to infrastructure investments, the project funded some important and highly cost-effective demand-side interventions under Component 1. These evidence-based interventions supported the growing health service demand, improved the quality of care and were at the core of PARSALUD I; a stronger focus on similar interventions would have further increased the cost-effectiveness of PARSALUD II.
    

Efficiency of project preparation and implementation

63. 
    Project preparation was characterized by delays leading to a three-year gap between the end of APL 1 and APL 2 effectiveness. Delays were largely due to the political economy in the country. Once the project gained political traction, it was rapidly appraised and negotiated.
    

64. 
    The project was implemented within the time period originally planned. The project was extended by 11 months, which balanced the 10-month gap between Bank approval and effectiveness. However, despite the extension, the project disbursed 80% of the planned amount, due to delays in procurement and civil works¹⁰.
    

65. 
    Given the nature of the project, the evaluation of Project efficiency should consider the Bank’s overall program financial contribution. In this sense, the project was extremely cost-effective, since by supporting less than 10% of the total program’s costs, it leveraged nearly US$ 140 million of domestic resources, in addition to the US$ 15 million provided by the IADB. Despite the initial civil works delays and the 80% final loan disbursement ratio, the program’s actual costs were fundamentally in line with the amount of resources identified at appraisal.
    

66. 
    The implementation efficiency of the overall Government program was leveraged by the project funds and the use of Bank fiduciary safeguards that provided cover for the weaker Government systems and processes. The Bank project team reviewed procurement bidding documents and contracts for civil works, consultancy services and provision of training for the overall program. This effort increased the leverage of technical and fiduciary implementation support beyond the loan funds and enhanced the technical skills of the PARSALUD technical and fiduciary team.
    

Fiscal impact and sustainability

67. 
    The fiscal impact of the project was marginal, limiting sustainability concerns. Project implementation did not have a major impact on the MINSA budget, as it weighed an average of 0.08 percent throughout the period analyzed. Once the facilities were built, improved or better equipped through the program’s funds, the responsibility for the management of those was transferred to the regional government. By signing the Convenios (contracts between PARSALUD and each of the nine regions), regional governments committed to allocate budget for maintenance of the upgraded facilities as a condition for starting civil works in pre-identified facilities in that specific region.
    

3.4 Justification of Overall Outcome Rating

The Overall Outcome Rating is: Moderately Satisfactory. This rating takes into account that the project is the second in a series of two APLs to support a broader GOP program that has been largely successful in achieving its goal of improving maternal and child health.

Table 6: Summary of Outcome Ratings

Project Outcome Ratings
Relevance	Substantial
Efficacy	Substantial
Efficiency	Modest
Overall Outcome Rating	Moderately Satisfactory

3.5 Overarching Themes, Other Outcomes and Impacts

1. 
   Poverty Impacts, Gender Aspects, and Social Development
   

68. 
    The project targeted specifically nine of the poorest regions, home of indigenous people and isolated communities, and within those communities focused on the most vulnerable groups ― women and children in the early years of life. The equity dimension was important in conceptualizing the project, which contributed to social inclusion and promoted a solid basis for personal identification and for the affiliation to the SIS. The project was mindful of local identities and practices and successfully built on those to select interventions that were culturally appropriate for a specific context, such as the parto vertical. This was key in ensuring that people were more comfortable in accessing the services, with the aim of sustaining improvements in access to basic services.
    

(b) Institutional Change/Strengthening

69. 
    The project showed progress in building capacity at the central level and most importantly at the regional and local levels, benefiting government representative and different categories of health professionals. For example, PARSALUD II contributed to the local and regional capabilities to conduct social dialogue with various stakeholders prior to project implementation. The prior consultations on project’s infrastructure (to build, improve and equip the health facilities for obstetric and neonatal care), was a coordinated effort involving indigenous populations, regional and local governments, which strengthened the ability of local and regional actors to seek consensus on health infrastructure building.
    

(c) Other Unintended Outcomes and Impacts (positive or negative)

70. 
    The project contributed to increased inclusion of indigenous communities and to provide space for communities and especially women to be heard, increasing accountability of the system. At the end of 2013, the project organized a South-South Knowledge workshop on intercultural interventions in health aimed to showcasing PARSALUD’s best lessons learned and bringing regional experiences that informed and strengthened PARSALUD’s intercultural agenda. The main attendants to the workshop were representatives of the Pan American Health Organization (PAHO), the National Autonomous University of Mexico, the Ministry of Health of Venezuela and the Ministry of Cultures of Peru, among others.
    

71. 
    The project had a pivotal role in fostering and guiding the identity rights movement in the health sector and in creating a platform for dialogue of different actors at the national and local levels. The systematic introduction of birth certificates as the first means of personal identification provided the legal basis for the registration to the SIS as well as other GOP programs and access to the related benefits.
    

72. 
    The project improved awareness of environmental issues, helped create a conducive institutional environment to ensure environmental hazards are considered and dealt with, and built capacity at the central, regional and local level.
    

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

Not available.