December, 2016 - SUPPORT Summary of a systematic review | print this article |

What are the impacts of teaching critical appraisal skills in healthcare settings?

Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity (closeness to the truth), results and relevance to an individual’s work. The goal of formal training in critical appraisal skills is to help healthcare workers in understanding results of research studies and their relevance to patient care. This review focused on training for qualified health professionals in practice, and not health professional students.

Key messages

  • Teaching critical appraisal skills to health professionals may improve their knowledge of how to critically appraise research papers.
  • It is uncertain whether teaching critical appraisal skills to health professionals leads to actual changes in their critical appraisal skills.
  • No study was found that evaluated the impact of teaching critical appraisal skills on processes of care or patient outcomes.
  • None of the included studies were from low-income countries.

 

Background

When healthcare professionals face a dilemma concerning the effectiveness of an intervention for their patients, one option is to locate and appraise relevant scientific studies. Formal training in critical appraisal skills may assist healthcare workers in assessing and interpreting the validity and relevance of research evidence. Critical appraisal may also help healthcare workers deal with information overload.


About the systematic review underlying this summary

Review objectives: To assess the effects of teaching critical appraisal skills to health professionals on the process of care, patient outcomes and knowledge of health professionals
Type of What the review authors searched for What the review authors found
Study designs & interventions Randomised trials, non-randomised trials, controlled before-after studies, and interrupted time series studies that examined the effectiveness of educational interventions teaching critical appraisal to health professionals
Three randomised trials of: journal club supported by a half-day workshop (1 study); critical appraisal materials ( a package including papers with methodological reviews), list-serve discussions and articles (1 study); and a half-day workshop based on a Critical Appraisal Skills Programme (CASP) (1 study)
Participants Any qualified healthcare professionals (including managers and purchasers) with direct patient care. Studies involving students were excluded
Interns (1 study) and physicians (2 studies)
Settings Any clinical setting
USA, UK and Canada
Outcomes Process of care, patient mortality, morbidity, quality of life and satisfaction
Knowledge (2 studies) and critical appraisal skills (3 studies)
Date of most recent search: June 2011
Limitations:This was generally a well-conducted systematic review with only minor limitations.

Horsley T, Hyde C, Santesso N, et al. Teaching critical appraisal skills in healthcare settings. Cochrane Database Syst Rev 2011; 9 (11): CD001270. 

Summary of findings

The review identified three studies involving 272 participants. None of the included studies were conducted in a low-income country.

 

  • Teaching critical appraisal skills to health professionals may improve their knowledge of how to critically appraise research papers. The certainty of this evidence is low.
  • It is uncertain whether teaching critical appraisal skills to health professionals leads to actual changes in their critical appraisal skills. The certainty of this evidence is very low.
  • None of the included studies evaluated the effects of teaching critical appraisal skills on processes of care or patient outcomes.

 

Teaching critical appraisal for improving process of care variables, patient outcomes and health professionals’ knowledge

Population           Health professionals

Settings               Any setting

Intervention        Teaching critical appraisal

Comparison         Usual care

Outcomes

Effects

No of participants (studies)

Certainty

of the evidence

(GRADE)

Process of care

Not reported

-

-

Patient-related outcomes

Not reported

-

-

Critical appraisal skills

Heterogenous measurement scales

Follow-up: 0 to 6 months

Effects on critical appraisal skills were uncertain

160

(3 studies)

Very low

Knowledge

Follow-up: 0 to 6 months

Critical appraisal training resulted in improvements in knowledge in both studies

146

(2 studies)

Relevance of the review for low-income countries

Findings Interpretation*
APPLICABILITY
None of the included studies were conducted in a low-income country. Interventions assessed included journal club supported by a half-day workshop, critical appraisal materials, listserve discussions and articles and a half-day Critical Appraisal Skills Programme (CASP) workshop

It is uncertain whether some of the interventions that were evaluated would be feasible in low-income countries.

Although impacts on knowledge might be similar, the certainty of this evidence was low, and the certainty of the evidence for other outcomes was very low.


EQUITY
The included studies did not report data regarding differential effects of critical appraisal educational interventions (workshops, materials and programmes) across populations with varying socioeconomic status.
The critical appraisal interventions assessed (workshops, materials, programmes) may require resources and technical skills that are often scarce in low-income settings; this may limit their utilisation and effectiveness in those settings.
ECONOMIC CONSIDERATIONS
None of the included studies assessed costs associated with critical appraisal educational interventions. In one study one-off workshops was estimated to cost GBP 250 per participant.
The potential benefits of critical appraisal educational interventions need to be weighed against the time, efforts and money associated with their implementation on a large scale (and gains from other uses of resources, such as direct investments in patient care).
MONITORING & EVALUATION
There was no eligible study on the effects of educational interventions teaching critical appraisal skills in healthcare settings in low-income countries.

Randomised trials of educational interventions teaching critical appraisal to health professionals in low-income countries are needed. Outcomes assessed should include professional practice, patient outcomes and costs.

 


*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see:
www.supportsummaries.org/methods


 

Additional information

Related literature

Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews. PLoS One 2014; 9(1): e86706.

 

Akl EA, Kairouz VF, Sackett KM, et al. Educational games for health professionals. Cochrane Database Syst Rev 2013; 3: CD006411.

 

Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ 2004; 329(7473):1017.

 

This summary was prepared by

Newton Opiyo, KEMRI-Wellcome Trust Research Programme, Kenya.

 

Conflict of interest

None declared. For details, see: www.supportsummaries.org/coi

 

Acknowledgements

This summary has been peer reviewed by: Chris Hyde, Nancy Santesso and Frode Forland.

 

This review should be cited as

Horsley T, Hyde C, Santesso N, et al. Teaching critical appraisal skills in healthcare settings. Cochrane Database Syst Rev 2011; 9 (11): CD001270.

 

The summary should be cited as

Opiyo N.  What are the impacts of teaching critical appraisal skills in healthcare settings? A SUPPORT Summary of a systematic review. December 2016. www.support-collaboration.org/summaries.htm

 

Keywords

evidence-informed health policy, evidence-based, systematic review, health systems research, health care, information literacy, low and middle-income countries, developing countries, primary health care



Comments