August, 2016 - SUPPORT Summary of a systematic review | print this article |

Does public disclosure of performance data improve quality of healthcare?

Information about the performance of health plans (health insurance or subscription-based medical care), hospitals, and healthcare professionals is increasingly available in the public domain. However, the effects of such public disclosure of performance data are unclear.

 

Key messages

  • Public disclosure of performance for health plans:
    • May lead to patients selecting health plans that have better quality ratings
    • Has uncertain effects on quality improvement activities
    • May improve health outcomes
  • Public disclosure of performance for hospitals:
    • May lead to little or no difference in patient selection of hospitals
    • Probably stimulates quality improvement activities
    • May improve health outcomes
  • Public disclosure of performance for individual healthcare providers:
    • Probably leads to patients selecting providers that have better quality ratings
    • Has uncertain effects on quality improvement activities
    • May improve health outcomes
  • All of the included studies were conducted in high-income countries.
    • Public disclosure of performance may be difficult to implement in low-income countries because of limitations of the ability of health facilities and providers to produce accurate data, the capacity to disseminate the data, the ability of patients to interpret the data and, in some places, patients not having multiple facilities or providers from which to choose.

Background

Public disclosure of performance data has been proposed as a mechanism for improving the quality of care by providing more transparency and by making healthcare providers more accountable. It is assumed that such public disclosures would lead patients to seek care from betterperforming healthcare providers and help healthcare providers to identify areas in which they are deficient and thus improve their performance. Examples of publicly disclosed performance data include mortality rates, patient satisfaction, length of stay, and immunisation coverage.



About the systematic review underlying this summary

Review objectivesTo synthesise the evidence for using public disclosure of performance data to improve healthcare quality
Type of What the review authors searched for What the review authors found

Study designs & interventions

Peer-reviewed articles published between 1986 and 2006. Type of studies not pre-specified

2 randomised trials, 2 non-randomised trials, 1 controlled before-after study, 9 interrupted time series studies, and 31 other observational studies.

Participants

Not pre-specified

Hospitals, patients, and hospital staff (45 studies)

Settings

Not pre-specified

USA (43 studies), United Kingdom (1), Canada (1)

Outcomes

Selection of health plans, hospitals, and individual providers, quality improvement activity, clinical outcomes, unintended consequences

Selection of health plans (8 studies), selection of hospitals (9), selection of individual providers (7), quality improvement activity (11), clinical outcomes (11), unintended consequences (13)

Date of most recent search: March 2006

Limitations: Only peer-reviewed, English-language articles were included.

Fung CH, Lim YW, Mattke S, et al. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008; 148:111-23.

Fung CH, Lim YW, Mattke S, et al. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008; 148:111-23.

 

Summary of findings

The review identified 45 studies conducted in high-income countries. The results are organised by reporting level (health plans, hospitals, individual providers).

1) Health plans

Ten studies assessed the effects of public disclosure of performance data on health plans and showed that:

  • Public disclosure may lead patients to select health plans with better quality ratings or to avoid those with worse ratings. The certainty of this evidence is low.
  • The impact of public disclosure on quality improvement activities is uncertain. This outcome was not reported.
  • Public disclosure may lead to slight improvements in clinical outcomes. The certainty of this evidence is low.

Effects of public disclosure on health plan quality

People:  Users of health insurance plans; health insurance plans
Settings
:  USA (10 studies)
Intervention
: Public release of performance data
Comparison
No intervention

Outcomes Impact Certainty of the evidence
(GRADE)

Selection of health plans


2 studies found no effect of public disclosure on health plan choices, 6 studies found that public disclosure may lead users to select better-rated plans or to avoid lower-scoring ones.


Quality
improvement activity

Quality improvement activity


No studies of health plan performance data and quality improvement activity were identified.


-

Clinical outcomes


1 study reported that technical performance measures and patient experience measures were higher for health plans with publicly disclosed data.



GRADE: GRADE Working Group grades of evidence (see above and last page)

 

2) Hospitals

Eighteen studies assessed the effects of the public disclosure of performance data on hospitals and showed that such public reporting:

  • May lead to little or no difference in patient selection of hospitals. The certainty of this evidence is low.
  • Probably stimulates hospitals to undertake quality improvement activities. The certainty of this evidence is moderate.
  • May lead to slight improvements in clinical outcomes. The certainty of this evidence is low.

 

Effects of public disclosure on quality of care provided in hospitals  

People:  Patients, hospitals, hospital staff
Settings
:  USA (16 studies), UK (1), Canada (1)
Intervention
: Public release of performance data
Comparison
: No intervention

Outcomes Impact Certainty of the evidence
(GRADE)

Selection of hospitals

 

3 studies reported that hospitals with better outcomes experienced higher rates of growth in market share, but 6 studies found no association.

 

 

Quality improvement activity

 

9 studies reported that public disclosure led to quality improvement activities; however, 2 studies showed only minimal impact on quality improvement activities.

 

 

Clinical outcomes

 

7 studies found decreases in mortality or improvement in other clinical outcomes; however, 4 studies did not find any impact on clinical outcomes.

 

 

GRADE: GRADE Working Group grades of evidence (see above and last page)

 

3) Individual healthcare providers

Eight studies assessed the effects of public disclosure of performance data on individual providers and showed that:

  • Public disclosure probably influences users of healthcare services to select providers with better quality ratings or to avoid those with worse ratings. The certainty of this evidence is moderate.
  • The impact of public disclosure on quality improvement activities among individual providers of healthcare is uncertain. This outcome was not reported.
  • Public disclosure of performance data may improve clinical outcomes among individual providers. The certainty of this evidence is low.

Effects of public disclosure on quality of care provided by individual healthcare providers

People:  Patients, physicians, or physician groups
Settings
:  USA (8 studies)
Intervention
: Public disclosure of performance data
Comparison
: No intervention

Outcomes Impact Certainty of the evidence
(GRADE)

Selection of providers


6 studies reported that public disclosure affected selection of health providers, while 1 study found no association.

 

Quality improvement activity


No published studies of the effect of public disclosure of performance data on quality improvement activity among physicians or physician groups were identified by the authors.

 

-

Clinical outcomes


1 study showed that risk-adjusted mortality rates for surgeons decreased after the data were released.

 

 

GRADE: GRADE Working Group grades of evidence (see above and last page)

 


 

 

 

Relevance of the review for low-income countries

 

 

www.supportsummaries.org/methods 

Findings Interpretation*
APPLICABILITY

The studies, all conducted in high-income countries, provided limited evidence of the benefits of public disclosure of performance data.


 

  • Public disclosure of performance data may be difficult to implement in low-income countries because of limitations of the ability of health facilities and providers to produce accurate data, the capacity to disseminate the data, the ability of patients to interpret the data and, in some places, patients not having multiple facilities or providers from which to choose. Consequently, the potential benefits are even more uncertain in low-income countries than in high-income countries.

 

 

EQUITY

In the post-report period of one study, it was noted that patients from more affluent neighborhoods were more likely to be treated by doctors with better quality ratings. Patients from poorer neighborhoods were more likely to be treated by doctors with worse ratings.

 

 

  • This limited evidence suggests either that the public disclosure of performance data may lead wealthier patients to select better performing doctors, that better doctors may charge higher fees which poor patients can’t afford, or both. Such situations may exacerbate healthcare inequalities.

 

ECONOMIC CONSIDERATIONS

None of the studies examined the costs or costeffectiveness of publicly disclosing performance data.

 

 

  • Both costing studies and cost-effectiveness studies are needed, in addition to more rigorus evaluations of the effects of public disclosure of performance data.

 

MONITORING & EVALUATION

There is important uncertainty about the effects of public disclosure of performance data, especially in low-income countries.


 

  • Randomised trials of public disclosure of performance data are needed in low-income countries before scaling up its use or, interrupted time series analyses should be planned and designed before implementing policies that require public disclosure of performance data.

*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see:

www.supportsummaries.org/methods

Additional information

Related literature

Faber M, Bosch M, Wollersheim H, et al. Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care 2009; 47:1-8.

 

Henderson A, Henderson S. Provision of a surgeon's performance data for people considering elective surgery. Cochrane Database of Systematic Reviews 2010, Issue 11. Art. No.: CD006327.

 

Ketelaar NABM, Faber MJ, Flottorp S, et al. Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations. Cochrane Database of Systematic Reviews 2011, Issue 11. Art. No.: CD004538.

 

Kolstad JT, Chernew ME. Quality and consumer decision making in the market for health insurance and health care services. Medical Care Research and Review 2009 ;66(Suppl 1):28S-52S.

 

This summary was prepared by

Charles Shey Wiysonge, Centre for Evidence-based Health Care, Stellenbosch University, & Cochrane South Africa, South African Medical Research Council, Cape Town, South Africa

 

Conflict of interest

None declared. For details, see: www.supportsummaries.org/coi

 

Acknowledgements

This summary has been peer reviewed by: Constance H. Fung, Hanna Bergman, and a reviewer who

prefers to remain anonymous

 

This review should be cited as

Fung CH, Lim YW, Mattke S, et al. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008; 148:111-23.

 

The summary should be cited as

Wiysonge CS. Does public disclosure of performance data improve quality of healthcare? A SUPPORT Summary of a systematic review. August 2016. www.supportsummaries.org

 

Keywords

evidence-informed health policy, evidence-based, systematic review, health systems research, health care, low and middle-income countries, developing countries, primary health care, public release of performance data

 

This summary was prepared with additional support from:

The University of Cape Town, South Africa

The University of Cape Town aspires to become a premier academic meeting point between South Africa, the rest of Africa, and the world. Taking advantage of expanding global networks and our distinct vantage point in Africa, we are committed, through innovative research and scholarship, to grapple with the key issues of our natural and social worlds.

www.uct.ac.za

 



Comments