SUPPORT Summaries
Evidence of the effects of health system interventions
for low- and middle-income countries
Evidence of the effects of health system interventions
for low- and middle-income countries
August, 2016 - SUPPORT Summary of a systematic review | print this article | download PDF
Safety checklists are used as tools to improve care processes and patient safety outcomes.
Key messages
People deciding on strategies to improve patient safety
Ko HC, Turner TJ, Finnigan MA. Systematic review of safety checklists for use by medical care teams in acute hospital settings--limited evidence of effectiveness. BMC Health Serv Res. 2011; 11:211
A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.
SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.
Guidance is available on how to create checklists, what should be included, and how to implement them. However, checklists are often implemented as a part of multicomponent quality improvement initiatives. It has been unclear whether checklists are effective in improving patient safety in acute care settings. To the extent that they are effective, it is unclear what checklist designs and implementation tools are most effective. It is also unclear to what extent checklists themselves contribute to the effectiveness of multicomponent interventions.
Safety checklists can be either paper-based or electronic. This summary is focused on paper-based checklists.
|
About the systematic review underlying this summary |
||
|
Review objectives: To assess if the use of safety checklists, compared to not using checklists, improves patient safety in acute hospital settings |
||
| Type of | What the review authors searched for | What the review authors found |
|---|---|---|
|
Study designs & interventions |
Comparative studies of paper-based checklists, applied to hospitalized patients by medical care teams, compared to controls (care provided without checklists). |
Before-after studies (9) that evaluated a wide variety of checklist designs and training on use of the checklists. |
|
Participants |
Medical care teams (a medical clinician or surgeon had to be included). |
Medical teams. |
|
Settings |
Acute hospital settings. |
Intensive care units (5 studies), emergency departments (2), surgical units (1) and multi-departmental acute care settings (1) |
|
Outcomes |
Any patient relevant clinical outcome. |
Length of stay (3 studies), percentage of ventilator days on which patients received recommended care (1), time from admission until prescription of medical deep venous thrombosis prophylaxis (1), appropriate indications for use of an indwelling urinary tract catheter (1), complications during the postoperative period (1), patients receiving antibiotics within eight hours of a diagnosis of pneumonia (1). |
|
Date of most recent search: September 2009 |
||
|
Limitations: Only articles in English were included and the results of included studies were not described or analysed systematically. |
||
Ko HC, Turner TJ, Finnigan MA. Systematic review of safety checklists for use by medical care teams in acute hospital settings--limited evidence of effectiveness. BMC Health Serv Res. 2011; 11:211
High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.
Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.
Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.
Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.
* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.
† Substantially different = a large enough difference that it might affect a decision.
See last page for more information.
The review included nine before-after studies. Most studies (eight) were done in North America and one study was done in eight countries (Canada, Jordan, India, New Zealand, Philippines, Tanzania, United Kingdom and United States). Four clinical settings were covered: five studies in the intensive care unit, two studies in the emergency department, one study in surgery, and one study in multi-departmental acute care.
1) Intensive care unit setting
Five studies conducted in the United States evaluated checklists in the intensive care unit setting. All studies had a high risk of bias, and given the important methodological differences between them, they cannot be summarised quantitatively.
|
Intensive care unit setting |
||||
|
People: Healthcare professionals Settings: Acute hospitals Intervention: Paper-based checklists Comparison: Care provided without checklists |
||||
| Outcomes |
Impact |
Number of studies |
Certainty of the evidence (GRADE) |
|
|
Length of stay |
Different checklists were used among studies. One of the studies found a reduction in the length of stay, but the other two did not. |
3 |
Very Low |
|
|
Percentage of ventilation days on which patient received four care process (Prophylaxis of peptic ulcer disease and deep venous thrombosis, appropriate sedation and recumbent positioning. |
During the period that the surgical checklist was used, the compliance in the four processes improved from 30% to 96% (p <0.001). |
1 |
Very Low |
|
|
Improvement in four domains (Use of physical therapy, transfer to telemetry, time from admission to the prescription of medical deep venous thrombosis prophylaxis, and central catheter duration) |
The use of the checklist was associated with an improvement in two of the four domains. |
1 |
Very Low |
|
|
GRADE: GRADE Working Group grades of evidence (see above and last page) |
||||
2) Emergency department setting
Two studies evaluated checklists in the emergency department. The included studies have a high risk of bias and they could not be summarised quantitatively.
|
Emergency department setting |
||||
|
People: Healthcare professionals Settings: Acute hospitals Intervention: Paper-based checklists Comparison: Care provided without checklists |
||||
| Outcomes |
Impact |
Number of studies |
Certainty of the evidence (GRADE) |
|
|
Length of stay. |
Post-endoscopy checklist after emergency department admission was used. The study found a reduction of 50% in the length of stay during the checklist period (p=0.003). |
1 |
Very Low |
|
|
Appropriate use of catheter in patients with indwelling urinary tract catheter. |
There was an increase of appropriate use of urinary tract catheters during the intervention period (from 37% to 51%; p=0.06). |
1 |
Very Low |
|
|
Documentation of an indication for a catheter in patients with indwelling urinary tract catheter. |
Documentation of an indication for a catheter remained unchanged during the intervention period. |
1 |
Very Low
|
|
|
Presence of a physician order for urinary tract catheter placement. |
The presence of a physician order increased from 43% to 63% post- intervention. |
1 |
Very Low |
|
|
GRADE: GRADE Working Group grades of evidence (see above and last page) |
||||
3) Surgery setting
One study conducted in eight countries (Canada, Jordan, India, New Zealand, Philippines, Tanzania, United Kingdom and United States) evaluated checklists in the surgery setting (7688 patients undergoing non-cardiac surgery).
|
Surgery setting |
||||
|
People: Healthcare professionals Settings: Acute hospitals Intervention: Paper-based checklists Comparison: Care provided without checklists |
||||
| Outcomes | Impact | Number of studies |
Certainty of the evidence (GRADE) |
|
|
Any major complication (including death) within the first 30 days after the operation. |
The rate of death declined from 1.5% to 0.8% during the intervention period. Complications also decreased from 11% to 7%. |
1 |
Low |
|
|
GRADE: GRADE Working Group grades of evidence (see above and last page) |
||||
4) Acute care setting
One study conducted in the United States evaluated checklists in the surgery setting (7688 patients undergoing non-cardiac surgery).
|
Acute care setting |
||||
|
People: Healthcare professionals Settings: Acute hospitals Intervention: Paper-based checklists Comparison: Care provided without checklists |
||||
| Outcomes |
Impact |
Number of studies |
Certainty of the evidence (GRADE) |
|
|
Proportion of patients receiving antibiotics within eight hours of a diagnosis of pneumonia. |
Hospitals using a checklist administered appropriate antibiotics more often than hospitals without the checklist (OR 2.0, 95% CI not reported p=0.0005). (Forms and reminders were used in addition to the checklist.) |
1 |
Very Low |
|
|
GRADE: GRADE Working Group grades of evidence (see above and last page) |
||||
| Findings | Interpretation* |
|---|---|
| APPLICABILITY | |
|
All studies except one were conducted only in high-income countries. One study included two low-income countries and found a different magnitude of the changes in outcomes before and after the intervention across study locations. |
The setting might influence the effectiveness of patient safety checklists. Those locations with good performance at baseline for the measured outcomes may have limited potential for improvements. |
| EQUITY | |
|
The study noted above investigated the use of checklists in more than one socio-economic and surgical setting. The authors noted no effects of income level or surgery type clusters on the outcomes. |
It is possible that resource levels, staff workloads, staff training and other factors could influence the effectiveness of patient safety checklists, and that they might be less effective in disadvantaged settings. |
| ECONOMIC CONSIDERATIONS | |
|
The studies did not include any economic evaluations. |
There may be some additional costs involved in training and educating staff on how to use checklists, as well as the time taken to use checklists. On the other hand, if they are found to improve safety, there may be savings. |
| MONITORING & EVALUATION | |
|
Included studies had a high risk of bias. |
Randomized trials are needed to evaluate the impacts of using checklists in acute care settings. |
|
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see: www.supportsummaries.org/methods |
|
Related literature
World Health Organization. Patient safety checklists. Available in http://www.who.int/patientsafety/implementation/checklists/en/
World Health Organization. Implementation manual WHO surgical safety checklist (first edition). Available in http://www.who.int/patientsafety/safesurgery/tools_resources/SSSL_Manual_finalJun08.pdf?ua=1.
Thomassen Ø, Storesund A, Søfteland E, Brattebø G. The effects of safety checklists in medicine: a systematic review. Acta Anaesthesiol Scand. 2014;58(1):5-18.
de Jager E, et al. Postoperative Adverse Events Inconsistently Improved by the World Health Organization Surgical Safety Checklist: A Systematic Literature Review of 25 Studies. World J Surg. 2016 Apr 28. PMID: 27125680
This summary was prepared by
Dimelza Osorio, Biomedical Research Institute Sant Pau - Iberoamerican Cochrane Centre, Barcelona, Spain.
Conflict of interest
None declared. For details, see: www.supportsummaries.org/coi
Acknowledgements
This summary has been peer reviewed by: Itziar Larizgoitia and Henry Ko
This review should be cited as
Ko HC, Turner TJ, Finnigan MA. Systematic review of safety checklists for use by medical care teams in acute hospital settings--limited evidence of effectiveness. BMC Health Serv Res. 2011; 11:211. Available at
http://www.biomedcentral.com/1472-6963/11/211.
The summary should be cited as
Osorio D. Do paper-based safety checklists improve patient safety in acute hospital settings? A SUPPORT Summary of a systematic review. August 2016.
Keywords
Evidence-informed health policy, evidence-based, systematic review, health systems research, health care, low and middle-income countries, developing countries, primary health care, safety checklists, medical checklists, patient safety, pa-per-based checklists.
