January, 2017 - SUPPORT Summary of a systematic review | print this article | download PDF

What are the effects of interventions to improve the use of systematic reviews in decision-making by health system managers, policy makers, or clinicians?

Anumber of interventions aiming to increase the use of systematic review evidence in decision making are currently in use. These include summaries of systematic reviews designed to improve the accessibility of the findings (“information products”) and changes to organisational structures, such as employing specialist groups to synthesise evidence in order to inform local decision making.

 Key messages

  • Summary of findings tables probably improve access to the main findings of a systematic review and may decrease inappropriate use of clinical interventions.
  • Mailing printed bulletins that summarise the findings of systematic reviews may improve professional practice.
  • Multifaceted interventions that include access to a database or information derived from systematic reviews and training to improve the use of that evidence by clinicians may have little or no effect on professional practice and patient outcomes.
  • Tailored messages and knowledge brokers together with access to a database of systematic reviews may have little or no effect on self-reported use of research evidence by public health departments, compared to access to the database alone.
  • The review found no studies of the effects of interventions targeted at health system managers or policymakers, and no studies from low income countries.

 

Background

Systematic reviews provide a transparent and robust summary of existing research. However, health system managers, national and local policymakers, and healthcare professionals can face several obstacles when attempting to utilise this evidence. These include dealing with a large volume of research evidence and difficulties making judgements about the applicability of evidence from systematic reviews. In an attempt to increase the use of systematic review evidence in decision making, a number of interventions have been developed.



About the systematic review underlying this summary

Review objectives:To identify and assess the effects of information products based on the findings of systematic review evidence, and organisational supports and processes designed to support the uptake of systematic review evidence by health system managers, policy makers, and healthcare professionals.
Type of What the review authors searched for What the review authors found
Study designs & interventions Randomised trials, interrupted time series studies and controlled before after studies of interventions intended to improve the uptake of evidence from systematic reviews in decision making
Eight studies:multifaceted interventions (2 cluster randomised trials); summary of findings tables (1 randomised trial); analgesic league table (1 randomised trial);organizational intervention (knowledge broker, access to systematic review repository, provision of tailored messages) (1 randomised trial); dissemination of printed bulletins (3 interrupted timeseries studies).
Participants Health system managers, policymakers and clinicians
Nurses (1 study); physicians and nurses (1); public health professionals (1); evidence based practice workshop participants (1); healthcare professionals and students (1); National Health Service (NHS) clinicians and decision makers (3)
Settings Any setting
UK (5 studies); Canada (1); Mexico and Thailand (1); setting not specified (1)
Outcomes Utilisation of research, acceptability of the way information was presented, knowledge, utilisation of healthcare resources, patient related outcomes. Utilisation of research (6 studies); utilisation of healthcare resources (costs) (3); knowledge (1 RCT); perceived understanding and ease of use (1); preferences and attitudes (1); patientrelated outcomes (pain management and use of analgesia) (2)
Date of most recent search: March 2012
Limitations:This is a well-conducted systematic review with only minor limitations.
Murthy L, Shepperd S, Clarke MJ, et al. Interventions to improve the use of systematic reviews in decision making by health system managers, policy makers and clinicians. Cochrane Database Syst Rev 2012; 9: CD009401.

Murthy L, Shepperd S, Clarke MJ, et al. Interventions to improve the use of systematic reviews in decision making by health system managers, policy makers and clinicians. Cochrane Database Syst Rev 2012; 9: CD009401.

Summary of findings

The review identified eight studies that evaluated the effectiveness of different interventions designed to support the use of systematic review evidence decision making.

1) Multifaceted interventions targeted at clinicians

  Three studies evaluated the effects of multifaceted interventions on obstetric practice (1 each in Mexico, Thailand and the UK). One study evaluated the effects of a multifaceted intervention on patient outcomes (pain and use of analgesics) for patients on four orthopaedic wards in the UK. The interventions included access to a database (3 studies) or a league table of analgesic efficacy (1 study), training (interactive workshops or educational outreach), audit and feedback (2 studies), and a coordinator to assist use of the database (2 studies).

  • Multifaceted interventions to improve the use of evidence from systematic reviews by clinicians may have little or no effect on professional practice and patient outcomes. The certainty of this evidence is low.

Multifaceted interventions to improve the use of evidence derived from systematic reviews in decision making by clinicians

People: Clinicians
Settings
:  Secondary care settings in the UK (2 studies), Mexico (1) and Thailand (1)
Intervention
: Multifaceted interventions to support the use of evidence derived from systematic reviews
Comparison
No intervention/usual care
Outcomes Median effect (range) Certainty of the evidence
(GRADE)

Professional practice

for six obstetric practices (2 studies), four obstetric practices (1)

Median effect 3.5% improvement in desired professional practice (adjusted for baseline differences)

(2.0% to 3.5%)

 


Low

Patient outcomes

pain and use of analgesics (1 study)

There was little or no difference in the average amount of pain or use of analgesics

Low

1 The size of the differences varied widely within each study and most of the effect estimates had wide confidence intervals.

2 There was a serious risk of bias and imprecision.

GRADE: GRADE Working Group grades of evidence (see above and last page)

2) Summaries of systematic reviews (information products) targeted at clinicians

 Three studies evaluated the effects of information products targeted at clinicians.

Summaries of systematic reviews targeted at clinicians:

  • probably increase the perceived accessibility of findings (moderate certainty evidence).
  • may decrease inappropriate use of clinical interventions (low certainty evidence).

Summaries of systematic reviews targeted at clinicians

People: General practitioners (1 study), surgeons (1), and diverse clinicians (1)
Settings
:  Primary care in the UK (1 study), secondary care in the UK (1), and a workshop in Norway (1)
Intervention
: A mailed summary of a systematic review (2 studies) and a summary of findings table (1) with a systematic review
Comparison
: No intervention/usual care (2 studies) and a systematic review without a summary of findings table (1)
Outcomes Effect Certainty of the evidence
(GRADE)

Professional practice

use of antidepressant drugs (1 study) and gromlet insertion (1 study)

Inappropriate use decreased

Low

Perceived accessibility of the findings

proportion that perceived the main findings to be very accessible -

6 or 7 on a scale from 1 (very inaccessible) to 7 (very accessible)


Difference 24% more

(41% versus 17%; p=0.037)

Moderate

1 Non randomized (interrupted time series) studies

2 Serious risk of bias

GRADE: GRADE Working Group grades of evidence (see above and last page

3) Interventions targeted at public health departments

 One study in Canada evaluated the added effects of tailored messages (one group) and tailored messages together with knowledge brokers (one group) together with access to a database of systematic reviews compared to access to the database alone.

  • Tailored messages with or without knowledge brokers may have little or no effect on self reported use of research evidence by public health departments.

 



 

 



Relevance of the review for low-income countries

Findings Interpretation*
APPLICABILITY
None of the included studies was conducted in a low income country. Evaluated interventions included passive dissemination of systematic review evidence (in the form of bulletins) and multifaceted interventions (which provide access to, and training in the use of systematic review evidence).
The impact of information products derived from systematic reviews (printed bulletins) and multifaceted interventions evaluated by the included studies is uncertain in low income countries.
EQUITY
The included studies did not report data regarding differential effects of systematic review information products and organisational processes (bulletins, multifaceted interventions) evaluated across different settings.

The interventions evaluated (summary of findings tables, league tables, knowledge brokers, systematic review repositories) require information systems and technical skills that may be lacking in low income settings. This may limit access and utilisation in low income settings.

ECONOMIC CONSIDERATIONS
No data were available from low income settings. Three studies in high income countries reported data on health resource utilisation (costs): implementation cost of a single educational visit, and costs of production / distribution and potential savings of bulletins summarising systematic review evidence.

Costs of interventions intended to improve the use of systematic review evidence in decision making include costs related to training, production, and dissemination of information products. Such costs may limit utilisation, and consequently effectiveness of these interventions in low income countries.

MONITORING & EVALUATION
There was no eligible study of interventions for improving use of research evidence in decision making processes in low income countries.

Randomised trials are needed to evaluate the effects of different interventions designed to support uptake of systematic review evidence in low income countries.


*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low income countries. For additional details about how these judgements were made see: www.supportsummaries.org/methods.

 

Additional information

Related literature

Tricco AC, Cardoso R, Thomas SM, et al. Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review. Implement Sci. 2016;11:4.

 

Wallace J, Byrne C, Clarke M. Making evidence more wanted: a systematic review of facilitators to enhance the uptake of evidence from systematic reviews and meta analyses. Int J Evid Based Healthc 2012;10(4):338-46.

 

Chambers D, Wilson PM, Thompson CA, et al. Maximizing the impact of systematic reviews in health care decision making: a systematic scoping review of knowledge translation resources. Millbank Quarterly 2011;89(1):131-56.

 

Perrier L, Mrklas K, Shepperd S, et al. Interventions encouraging the use of systematic reviews in clinical decision making: a systematic review. J Gen Intern Med 2011;26(4):419-26.

 

Perrier L, Mrklas K, Lavis J, Straus S. Interventions encouraging the use of systematic reviews by health policymakers and managers: a systematic review. Implementation Sci 2011;6:43.

 

Dobbins M, Cockerill R, Barnsley J. Factors affecting the utilization of systematic reviews. A study of public health decision makers. Int J Technol Assess Health Care 2001;17(2):203–14.

 

This summary was prepared by

Newton Opiyo, KEMRI Wellcome Trust Research Programme, Kenya.

Conflict of interest

None declared. For details, see: www.supportsummaries.org/coi

Acknowledgements

This summary has been peer reviewed by Airton Stein. We did not receive any comments from the review authors.

This review should be cited as

Murthy L, Shepperd S, Clarke MJ, et al. Interventions to improve the use of systematic reviews in decision making by health system managers, policy makers and clinicians. Cochrane Database Syst Rev 2012; 9: CD009401.

The summary should be cited as

Opiyo N. What are the effects of interventions to improve the use of systematic reviews in decision making by health system managers, policy makers and clinicians? A SUPPORT Summary of a systematic review. January 2017. www.supportcollaboration.org/summaries.htm

Keywords

evidence informed health policy, evidence based, systematic review, health systems research, health care, low and middle income countries, developing countries, primary health care.

 

 



Comments