February, 2017 - SUPPORT Summary of a systematic review | print this article |

Does providing healthcare professionals with data about their performance improve their practice?1

Audit and feedback is commonly used as a strategy to improve professional practice. It appears logical that healthcare professionals would be prompted to modify their practice if given feedback that their clinical practice was inconsistent with that of their peers or accepted guidelines.

 

Key messages

 

  • Interventions that include audit and feedback (alone or as a core component of a multifaceted intervention) probably improve professionals’ adherence to desired practice compared with usual care.
  • Audit and feedback may be more effective when baseline professional performance is low; when the source of the feedback is a supervisor or senior colleague; when the feed back is delivered at least monthly; when it is provided both verbally and in a written format; and when it includes both explicit targets and an action plan.
  • The effects on patient outcomes of interventions that include audit and feedback may range from little if any effect to some improvement, compared with usual care.
  • Few randomised trials of audit and feedback in low income countries were found. Audit and feedback is difficult to implement if reliable, routinely collected data are not readily available.

 

Background

Audit and feedback can be defined as any summary of clinical performance of healthcare over a specified period of time that is fed back to healthcare providers with the aim of improving practice or the organisation of care. Feedback can be given in a written, electronic or verbal format. The summary of clinical performance may also include recommendations for clinical action.

As audit and feedback is used widely within healthcare organisations, it is important to consider its likely effects on clinical performance.

 



About the systematic review underlying this summary

Review objectives: To assess the effects of audit and feedback on the practice of healthcare professionals and on patient outcomes.
Type of What the review authors searched for What the review authors found
Study designs & interventions

Randomised trials assessing the effects of audit and feedback. Interventions were only included if audit and feedback was a core or essential element.

140 randomised trials were included. The interventions used were highly heterogeneous with respect to their content, format, timing and source.

Targeted behaviours were prescribing (39 trials), management of patients with diabetes or cardiovascular diseases (34), and test ordering (31). The remaining trials varied widely in terms of health conditions and targeted behaviours.

 


Participants Healthcare professionals responsible for patient care.
In most of the trials the healthcare professionals were physicians. Other targeted providers included dentists (1 trial), nurses (3), pharmacists (2), and a mix of providers (14).
Settings Healthcare settings

USA (69 trials), Canada (11), UK or Ireland (21), Australia or New Zealand (10), and elsewhere (29). Only 5 studies were conducted in low and middle income countries: Sudan (2), Thailand (1), Laos (1), Argentina and Uruguay (1).

94 trials were in outpatient settings, 36 in inpatient settings, and the clinical setting was unclear in 10 trials.

 


Outcomes Objectively measured provider performance or healthcare outcomes
There was large variation in outcome measures, and many trials reported multiple primary out comes. Most trials measured professional practice, with some also reporting patient outcomes.
Date of most recent search: December 2010
Limitations: This is a well conducted systematic review with only minor limitations.

Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2012; (6): CD000259.

Summary of findings

The review included 140 trials. Most trials were conducted in high income countries (136). Only five trials were conducted in low and middle income countries (two in Sudan and one each in Thailand, Laos, and Argentina and Uruguay).

 The interventions varied in terms of content, format, timing, and source. In 121 trials, audit and feedback was targeted at physicians. In 91 trials one or more groups received a multifaceted intervention where audit and feedback was considered the core, essential component.

 Many trials reported multiple primary outcomes. Most trials reported professional practice outcomes, such as prescribing or use of laboratory tests, while some trials also reported patient outcomes, such as smoking status or blood pressure.


 1) Audit and feedback (with or without other interventions) compared to usual care

There was important heterogeneity in the results across trials.

 

  • Interventions that include audit and feedback probably improve professionals’ adherence to desired practice, compared with usual care. The certainty of this evidence is moderate.
  • The effects on patient outcomes of interventions that include audit and feedback may vary from little if any effect to some improvement, compared with usual care. The certainty of this evidence is low.
  • The effects of audit and feedback appear to vary based on the way the intervention is designed and delivered. Audit and feedback may be more effective when baseline professional performance is low; when the source of the feedback is a supervisor or senior colleague; when the feedback is delivered at least monthly; when it is provided both verbally and in a written format; and when it includes both explicit targets and an action plan.

Audit and feedback compared to usual care

People: Health care professionals.
Settings
:  Primary and secondary care in high, middle and low income countries.
Intervention
Audit and feedback with or without other interventions.
Comparison
Usual care.
Outcomes Impact (weighted absolute improvement or decrease)1 No of Participants
(studies)
Quality of the evidence
(GRADE)
Adherence to desired practice (dichotomous outcomes)
Median absolute increase in desired practice: 4.3% (IQR 0.5% to 16.0%)

82 comparisons from 49 studies2

[2310 clusters/groups of health providers (from 32 cluster trials) and 2053 health professionals (from 17 trials allocating individual providers)]

 

Moderate

 


Adherence to desired practice (continuous outcomes)
Median percent change in desired practice: 1.3% (IQR 1.3% to 28.9%)

26 comparisons from 21 studies

[661 clusters/groups of health providers (from 13 cluster trials) and 605 health professionals (from 8 trials allocating individual providers)]

 

Moderate

 


Patient outcomes (dichotomous) Median absolute decrease in desired outcomes: 0.4% (IQR -1.3% to 1.6%) 12 comparisons from 6 studies

 

Low

 

Patient outcomes (continuous) Median percent change in desired outcomes: 17% (IQR 1.5 to 17%) 8 comparisons from 5 studies

 

Low

 

IQR: Interquartile range GRADE: GRADE Working Group grades of evidence (see above and last page)

1.The post intervention risk differences are adjusted for pre intervention differences between the comparison groups to account for baseline differences. The effect was weighted across studies by the number of health professionals involved in the study to ensure that small trials did not contribute as much to the estimate of effect as large trials.

2.Many studies had more than two arms and therefore contributed multiple comparisons of audit and feedback versus usual care.

2) Audit and feedback compared to other interventions

A total of 22 comparisons from 20 trials were included in this analysis.

 

  • The effects of audit and feedback on adherence with desired practice or patient outcomes, when compared to other implementation strategies (e.g. reminders, educational outreach, case management, financial incentives, patient mediated interventions) are mixed.

 

 

 

 

Relevance of the review for low-income countries

Findings Interpretation*
APPLICABILITY

The 140 randomised trials reviewed covered an extensive range of interventions and settings, but only five of the studies were conducted in low and middle income countries.

 

  • The review findings are based on trials in which the levels of organization and support were potentially higher than those available outside of research settings.

Decisions about if and how to use audit and feedback to improve professional practice should be guided by pragmatic factors and local circumstances, including whether:

-The known or anticipated baseline adherence to guidelines or recommended practice is low

-Conducting regular audits is feasible and the resources needed to collect these data are low

-The data available for audit purposes are reliable

-Supervisors are available to provide feedback and to link this to an action plan

 

  • The scarcity of health professionals, low staff morale and motivation and resource constraints may limit the feasibility and potential of audit and feedback interventions in some settings.
EQUITY
Overall, the included studies provided little data regarding differential effects of the interventions for disadvantaged populations.
The resources needed to implement audit and feedback may be less easily available in disadvantaged settings.
ECONOMIC CONSIDERATIONS
Few trials reported the cost or cost effectiveness of the interventions.

The resources needed to implement audit and feedback are likely to vary across settings and need to be estimated based on specific local conditions, including the availability of reliable routinely collected data and personnel costs.

 

  • Providing adequate support to programmes for audit and feedback is likely to be vital to ensure effectiveness when scaling up the intervention.
MONITORING & EVALUATION

Few rigorous studies of the effects of audit and feedback have been conducted in low or middle income countries.

 

  • The effects of audit and feedback are likely to vary based on the way the intervention is designed and delivered.

The implementation of audit and feedback in low income settings should be accompanied by rigorous evaluation.

Future studies of audit and feedback should compare different ways of providing feedback and their resource implications.


*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low income countries. For additional details about how these judgements were made see:www.supportsummaries.org/methods

 


 

Additional information

Related literature

Ivers NM, Sales A, Colquhoun S, et al. No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci 9(1):14.

Ivers NM, Grimshaw JM, Jamtvedt G, et al. Growing Literature, Stagnant Science? Systematic Review, Meta Regression and Cumulative Analysis of Audit and Feedback Interventions in Health Care. J Gen Intern Med 2014; 29:1534-41.

Gardner B, Whittington C, McAteer J, et al. Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback. Soc Sci Med 2010; 70: 1618–25.

Hysong SJ. Meta-Analysis: audit and feedback features impact effectiveness on care quality. Med Care 2009; 47: 356–63.

NorthStar is a tool that provides a range of information, checklists, examples and tools on how to best design and evaluate quality improvement interventions. https://www.dropbox.com/sh/8mn8co0au0eupt1/AAC55wRS6YwLPRDsMWuTtueda?dl=0

 

This summary was prepared by

Tomás Pantoja, Department of Family Medicine, Pontificia Universidad Católica de Chile, Chile and Signe Flottorp, Norwegian Institute of Public Health, Oslo, Norway

 

Conflict of interest

None declared. For details, see: www.supportsummaries.org/coi

 

Acknowledgements

This summary has been peer reviewed by: Airton Stein and Noah Ivers.

 

This review should be cited as

Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2012; (6): CD000259.

 

The summary should be cited as

Pantoja T, Flottorp S. Does providing healthcare professionals with data about ther performance improve their practice? A SUPPORT Summary of a systematic review. February 2017. www.supportsummaries.org

 

Keywords

evidenceinformed health policy, evidence based, systematic review, health systems research, health care, low and middleincome countries, developing countries, primary health care, audit, feedback, audit and feedback.

 

 

 

 

 

 

 



Comments