SUPPORT Summaries
Evidence of the effects of health system interventions
for low- and middle-income countries
Evidence of the effects of health system interventions
for low- and middle-income countries
December, 2016 - SUPPORT Summary of a systematic review | print this article | download PDF
The adoption of the Paris Declaration on Aid Effectiveness in 2005 represented a global agenda to improve aid management and delivery. Its impact on maternal and reproductive health, particularly on the relevant Millenium Development Goals (MDGs), is an important issue in terms of global health strategies for low income countries. This review evaluates the impact of the Paris Declaration by assessing available evidence of how aid delivered under the Paris Principles impacts on development outcomes, focusing specifically on maternal and reproductive health (MDG 5).
Key messages
People making decisions concerning international aid for maternal and reproductive health.
Hayman R, Taylor EM, Crawford F, et al. The impact of aid on maternal and reproductive health: a systematic review to evaluate the effect of aid on the outcomes of Millennium Development Goal 5. London: EPPI Centre, 2011.
A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.
SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.
The Millennium Development Goal 5 (MDG 5) aimed to improve maternal and reproductive health outcomes by: a) reducing the maternal mortality ratio by 75%; and b) achieving universal access to reproductive healthcare, by 2015. The adoption of the Paris Declaration on Aid Effectiveness in 2005 represented a global agenda to improve aid management and delivery, partly in order to address the slow progress towards the MDGs. The principles can be summarised as: Ownership – developing countries set their own development strategies; Alignment – donor countries and organisations bring their support in line with these strategies and use local systems to deliver that support; Harmonisation – donor countries and organisations coordinate their actions, simplify procedures and share information to avoid duplication; Managing for Results – developing countries and donors focus on producing and measuring results; and Mutual Accountability – donors and developing countries are accountable for development results.
| Review objectives: To compare the effects on Millennium Development Goal 5 outcomes of aid delivered under the Paris Principles and aid delivered outside this framework. | ||
| Type of | What the review authors searched for | What the review authors found |
|---|---|---|
| Study designs & interventions |
Studies had to present empirical research (qualitative or quantitative), i.e. contain primary dataInterventions: aid delivered under the Paris Principles, aid in general, or directly comparing both. |
Interrupted time series (1 study), pre test post test (17), secondary data analysis (5), process training methodology (1), retrospective analyses (3), unclear (1), and qualitative components (3) 10 studies for aid delivered under the Paris Principles, and 20 for aid in general |
| Participants | Donors and receiving developing countries |
Bilateral donor agencies: USAID (8 studies); Canadian International Development Agency (1); DFID (4). Multilateral agencies: World Bank (8). Large number of donors (5). Non governmental organisations: CARE (1); Bill and Melinda Gates Foundation (2); Save the Children Australia (1); PEPFAR (1); MotherCare (1) |
| Settings | Studies had to refer to developing countries or regions. |
China (3), Honduras (2), Indonesia (3), Uzbekistan (1), Egypt (3), Nicaragua (1), Botswana (1), South Africa (1), People’s Democratic Republic of Lao (1), Tanzania (2), Cameroon (1), Bangladesh (2), Nepal (2), Ghana (2), Uganda (1), Madagascar (1), India (1), Pakistan (1), Guinea (1), Burkina Faso (1), Timor Leste (1), Rwanda (1), Zimbabwe (1) |
| Outcomes | Maternal mortality ratios, births attended by skilled birth personnel, contraception prevalence, adolescent birth rate, antenatal care coverage, unmet need for family planning, trends in maternal and reproductive health |
Maternal mortality ratio or rate (MDG 5.1): 12 studies Births attended by skilled birth personnel (MDG 5.2): 17 studies Contraceptive prevalence (MDG 5.3): 15 studies Adolescent birth rate (MDG 5.4): 1 study Antenatal care coverage (MDG 5.5): 14 studies Unmet need for family planning (MDG 5.6): 2 studies |
| Date of most recent search: August 2010 | ||
| Limitations: This review has important limitations. | ||
Hayman R, Taylor EM, Crawford F, et al. The impact of aid on maternal and reproductive health: a systematic review to evaluate the effect of aid on the outcomes of Millennium Development Goal 5. London: EPPI Centre, 2011.
High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.
Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.
Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.
Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.
* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.
† Substantially different = a large enough difference that it might affect a decision.
See last page for more information.
The review included 30 studies from 23 developing countries, including aid delivered under the Paris Principles (10 studies) and for aid in general (20 studies).
1) Aid delivered under the Paris Principles
The 10 studies covered a mixture of aid modalities, including budget support, sector budget support, multidonor trust funds, soft loans, silent partnerships, pooled funding and projects.
Aid delivered under the Paris Principles. |
||||
| People: Donors and receiving developing countries. Settings: Bangladesh, China, Egypt, Ghana, Honduras, Indonesia, Tanzania, Lao People’s Democratic Republic. Intervention: Aid assumed to be compliant with the Paris Principles* Comparison: No aid |
||||
| Outcomes | Impact | Certainty of the evidence (GRADE) |
||
| Maternal mortality ratios |
The impact on maternal mortality ratios of aid delivered under the Paris Principles is uncertain. |
Very low
|
||
| Proportion of births attended by skilled personnel |
The impact on births attended by skilled personnel of aid delivered under the Paris Principles is uncertain. |
Very low
|
||
| Contraceptive use | The impact on contraceptive use of aid delivered under the Paris Principles is uncertain. |
Very low
|
||
| Adolescent birth rate | No studies found by the review. | |||
| Antenatal care coverage | The impact on antenatal care coverage of aid delivered under the Paris Principles is uncertain. |
Very low
|
||
| Unmet need for family planning | The impact on unmet need for family planning of aid delivered under the Paris Principles is uncertain. |
Very low
|
||
|
GRADE: GRADE Working Group grades of evidence (see above and last page) * The activities in the included studies started before the Paris Declaration and are assumed to be compliant with the Paris Principles |
||||
The 20 studies included for this intervention did not adhere or it was not possible to ascertain their adherence to the Paris Principles.
Aid in general |
||||
| People: Donors and receiving developing countries. Settings: China, Indonesia, Uzbekistan, Egypt, Nicaragua, Botswana, South Africa, Tanzania, Cameroon, Bangladesh, Nepal, Ghana, Uganda, Madagascar, India, Pakistan, Guinea, Burkina Faso, Timor Leste, Rwanda, Zimbabwe. Intervention: Aid in general. Comparison: No aid |
||||
| Outcomes | Impact | Certainty of the evidence (GRADE) |
||
| Maternal mortality ratios |
The impact on maternal mortality ratios of aid delivered with no conformity with the Paris Principles is uncertain. |
Very low |
||
| Proportion of births attended by skilled personnel |
The impact on births attended by skilled personnel of aid delivered with no conformity with the Paris Principles is uncertain. |
Very low |
||
| Contraceptive use | The impact on contraceptive use of aid delivered with no conformity with the Paris Principles is uncertain. |
Very low |
||
| Adolescent birth rate | The impact on adolescent birth rate of aid delivered with no conformity with the Paris Principles is uncertain. |
Very low
|
||
| Antenatal care coverage | The impact on antenatal care coverage of aid delivered with no conformity with the Paris Principles is uncertain. |
Very low
|
||
| Unmet need for family planning | The impact on unmet need for family planning of aid delivered with no conformity with the Paris Principles is uncertain. |
Very low
|
||
| GRADE: GRADE Working Group grades of evidence (see above and last page) |
||||
| Findings | Interpretation* |
|---|---|
| APPLICABILITY | |
|
Many of the studies were conducted in low-income countries.
|
Local management or control capacities in low income countries might influence the assurance of the Paris Principles application in practice.
|
| EQUITY | |
| No equity outcome or considerations were explored in the review. |
If health programmes or interventions supported by aid do not consider impacts on equity, they could increase inequities. |
| ECONOMIC CONSIDERATIONS | |
| No economic related findings were explored in the review. Socioeconomic determinants were only considered as confounding factors in the analyses. |
Aid programmes should consider whether the amount and length of aid is sufficient to achieve intended goals and the economic sustainability of interventions after the aid ends. |
| MONITORING & EVALUATION | |
| The certainty of the evidence for aid interventions in low income countries is very low. |
Future aid supported interventions in maternal and reproductive health need to have a well structured evaluation plan aiming to explore the effects on important outcomes, including the impact on other areas of the health system, families’ lives and society when relevant. As randomised trials might be difficult to carry out, interrupted time series studies should be considered, with sufficient baseline data, detailed descriptions of the aid intervention characteristics (including the amounts of aid and country internal funding and the length of funding), the socio political context, and the primary outcomes. |
|
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low income countries. For additional details about how these judgements were made see:www.supportsummaries.org/methods |
|
Hsu J, Berman P, Mills A. Reproductive health priorities: evidence from a resource tracking analysis of official development assistance in 2009 and 2010. Lancet. 2013 May 18; 381 (9879):1772-82.
Hsu J, Pitt C, Greco G, Berman P, Mills A. Countdown to 2015: changes in official development assistance to maternal, newborn, and child health in 2009-10, and assessment of progress since 2003. Lancet. 2012 Sep 29; 380 (9848):1157-68.
Grépin KA, Leach Kemon K, Schneider M, Sridhar D. How to do (or not to do). Tracking data on development assistance for health. Health Policy Plan. 2012 Sep; 27 (6):527-34.
Piva P, Dodd R. Where did all the aid go? An in depth analysis of increased health aid flows over the past 10 years. Bull World Health Organ. 2009 Dec; 87 (12):930-9.
Ravishankar N, Gubbins P, Cooley RJ, Leach Kemon K, Michaud CM, Jamison DT, Murray CJ. Financing of global health: tracking development assistance for health from 1990 to 2007. Lancet. 2009 Jun 20; 373 (9681):2113-24.
Cristian Herrera. Department of Public Health and Evidence Based Healthcare Program. Pontificia Universidad Católica de Chile. Chile.
None declared. For details, see: www.supportsummaries.org/coi
This summary has been peer reviewed by: Rachel Hayman and Paolo Piva.
Hayman R, Taylor EM, Crawford F, et al. The impact of aid on maternal and reproductive health: a systematic review to evaluate the effect of aid on the outcomes of Millennium Development Goal 5. London: EPPI Centre, 2011.
Herrera C. Does international aid improve maternal and reproductive health? A SUPPORT Summary of a systematic review. December 2016. www.supportsummaries.org
evidence informed health policy, evidence based, systematic review, health systems research, health care, low income countries, developing countries, primary health care, international aid, donors, maternal and reproductive health.
The Unit for Health Policy and Systems Research of the Evidence Based Health-care Program, Faculty of Medicine, Pontificia Universidad Católica de Chile. Chile.
