November, 2016 - SUPPORT Summary of a systematic review | print this article | download PDF

Do external inspections of compliance with standards improve quality of care in healthcare organisations?

External inspection systems are used in healthcare to improve adherence to quality standards. They are intended to promote changes in organizational structures or processes, in healthcare provider behavior and consequently in patient outcomes.

Key messages

  • It is uncertain whether external inspection results in improved compliance with accreditation standards, improved quality of care or decreased healthcare-acquired infection (i.e. MRSA) rates in hospitals.
  • This review found no direct evidence on the effectiveness of external inspections of compliance with standard in ambulatory settings. We are therefore uncertain of the effects in this setting.
  • This review found no direct evidence on the effectiveness of external inspections of compliance with standards in low-income countries.

 

 

Background

An external inspection has been defined as “a system, process or arrangement in which some dimensions or characteristics of a healthcare provider organisation and its activities are assessed or analysed against a framework of ideas, knowledge, or measures derived or developed outside that organisation”. It is used within healthcare settings to promote improvements in the quality of care, changes in organizational structures or processes, in healthcare provider behavior and thereby in patient outcomes.

 

 



About the systematic review underlying this summary

Review objectives:To evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes.
Type of What the review authors searched for What the review authors found
Study designs & interventions Randomised trials, non-randomised trials, interrupted time series studies and controlled before-after studies evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes
1 cluster randomised trial conducted in South Africa and 1 before-after study reanalysed as an interrupted time series study, conducted in England. The study in South Africa assessed the effects of external inspection on compliance with hospital accreditation standards. The study conducted in England assessed the effects of the Healthcare Commissions Infection Inspection program on compliance with standards related to healthcare-acquired infections.
Participants Hospitals, primary healthcare organisations and other community-based healthcare organisations containing health professionals
20 public hospitals in Kwa Zulu province of South Africa, and all acute hospital trusts in England
Settings Any health system
1 study was conducted in South Africa and 1 in England
Outcomes Measures of healthcare organisational change (e.g. organisational performance, waiting list times, inpatient hospital stay time); measures of healthcare professional behaviour (e.g. referral rate, prescribing rate); measure of patient outcomes (e.g. mortality and condition-specific measures)
Outcomes assessed in one study were related with adherence to standards in: medical records, patient outcomes such as satisfaction and patient education, and outcomes related with health processes. The other study assessed the rate of hospital-acquired infections.
Date of most recent search: May 2011
Limitations: This is a well-conducted systematic review with only minor limitations.

Flodgren G, Pomey MP, Taber SA, Eccles MP. Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes. Cochrane Database of Systematic Reviews2011, Issue 11

Summary of findings

This review included two studies, one conducted in South Africa and the other in England. The South African study assessed the effects of external inspection on the compliance with national hospital accreditation standards defined by COHSASA (the Council for Health Services Accreditation for South Africa). The study conducted in England assessed the effects of the Healthcare Commissions Infection Inspection Program on compliance with standards for the prevention of healthcare-acquired infections.

1) External inspection of compliance with COHSASA hospital accreditation standards compared with no intervention

The COHSASA accreditation system has 6000 indicators for assessing hospital services. A subgroup of 421 indicators, which were considered as critical quality criteria, was analysed separately. An additional subgroup of 8 indicators of quality of care in hospitals was measured by the study that included: (i) nurses perception of clinical quality, participation and teamwork; (ii) patient satisfaction with care; (iii) patient medication education; (iv) medical records: accessibility and accuracy; (v) medical records: completeness; (vi) completeness of peri-operative notes; (vii) completeness of ward stock medicine labeling and (viii) hospital sanitation.

  • It is uncertain whether external inspection of compliance with COHSASA accreditation standards improves quality of care in hospitals in South Africa because the certainty of this evidence is very low.

 

External inspection of compliance with COHSASA hospital accreditation standards

People            Health professionals and patients in 20 hospitals

Settings          Kwa Zulu Region, South Africa secondary care

Intervention   External inspection of compliance with accreditation standards and performance related to indicators for hospital quality of                         care

Comparison     No intervention

Outcomes Impact Certainty
of the evidence
GRADE)

Compliance with COHSASA

accreditation standards

The total compliance score with the accreditation standards was greater in intervention hospitals compared with control hospitals

Very low

Compliance with COHSASA

accreditation standards - subgroup of critical criteria analysed

The compliance score for a sub-section of predefined critical criteria, deemed crucial for the function of the service elements, was greater in intervention hospitals compared with control hospitals

Very low

Indicators for hospital quality of care  

The performance as measured by indicators of hospital quality of care was similar in intervention hospitals compared with control hospitals

Very low

GRADE: GRADE Working Group grades of evidence (see above and last page)

2) External inspection of compliance with the ’Code of Practice’ and the law related to healthcare-acquired infections

The Code of Practice and the Healthcare Act, 2006 are used as standards in the Healthcare Commission’s Inspections Programme in England which aims to decrease the number of healthcare-acquired infections. In this study only data on rates of hospital-acquired methicillin-resistant Staphylococcus Aureus (MRSA) infections, one of the most frequent bacteria producing health-acquired infections in hospital setting, had a sufficient number of measurements before and after the intervention to allow re-analysis as an interrupted time series.

  • It is uncertain whether external inspection of compliance with the “Code of Pratice” and the law related to health-acquired infections in hospitals in England decreases MRSA infection rate because the certainty of this evidence is very low.

External inspection of compliance with the ’Code of Practice’ and the law related to healthcare-acquired infections for improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes

People              Health professionals and patients in hospitals

Settings            Hospitals in England

Intervention     External inspection of compliance with the Code of Practice and the Health Care Act related to healthcare-acquired                                      infections

Comparison      No external inspection (in the same hospitals prior to the intervention)

Outcomes Impact Certainty
of the evidence
GRADE)

MRSA infection rate

Re-analysis of the quarterly reported rate of MRSA

cases, as an interrupted time series, did not indicate an impact of the Healthcare Commission’s Infection Inspection Programme.

Very low

GRADE: GRADE Working Group grades of evidence (see above and last page)

MRSA: methicillin-resistant Staphylococcus Aureus

Relevance of the review for low-income countries

Findings Interpretation*
APPLICABILITY

Neither of the two studies included in this review were conducted in a low-income country: one was done in South Africa and the other in England.

Both studies assessed the effect of external inspection of compliance with different standards on quality of hospital services, and one on hospital-acquired infection rates.

According to the findings in this review, it is uncertain whether external inspection contributes to improving quality of health services in hospital settings.

External inspection of compliance with standards may have varying acceptability and impact across different healthcare and cultural settings; may involve different components from training to organisational restructuring; and may impact in different ways on consumer and provider satisfaction across different settings.

Although quality of care is an objective of care in all health systems, it is not possible to be confident about the applicability of the reported interventions to low-income countries and to settings other than hospital care.


EQUITY
The included trials did not provide data regarding differential effects of the interventions for disadvantaged populations.
The resources needed to implement an external inspection of compliance with standards could affect provision of services in areas with fewer resources for healthcare. However if it resulted in improvements in patient health outcomes, it could lead to more benefits in disadvantaged populations.
ECONOMIC CONSIDERATIONS
The studies did not include direct evidence of the costs or cost-effectiveness of external inspection of compliance with care standards in hospital settings.

The cost of different types of external inspection of compliance with care standards is likely to be highly variable and costs must be estimated based on the specific intervention and local conditions outside of research settings.

If external inspection results in improved healthcare behaviours and outcomes, then these programmes might result in cost savings


MONITORING & EVALUATION
The certainty of the evidence is very low and no evidence from ambulatory care settings or low-income countries was identified.

Cluster randomised trials or interrupted time series studies would be the best study designs to assess the effects of these interventions. These studies should assess the effects of external inspection in different settings and consider relevant patient outcomes and use of resources.


*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see:
www.supportsummaries.org/methods


 

Additional information

Related literature

Walsche K, Freeman T, Latham L, Wallace L, Spurgeon P. Chapter 6. The development of external reviews of clinical governance. Clinical governance - from policy to practice. Birmingham, UK: University of Birmingham, Health Services Management Centre, 2000.

 

This summary was prepared by:

Blanca Penaloza,  Unit for Health Policy and System Research, Program of Evidence Based Health Care, Pontificia Universidad Catolica de Chile.

 

Conflict of interest

None declared. For details, see: www.supportsummaries.org/coi

 

Acknowledgements

This summary has been peer reviewed by: Gerd Flodgren and an anonymous referee.

 

This review should be cited as

Flodgren G, Pomey MP, Taber SA, Eccles MP. Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes. Cochrane Database of Systematic Reviews 2011, Issue 11.

 

The summary should be cited as

Penaloza B. Do external inspections of compliance with standards improve quality of care in healthcare organisations? A SUPPORT Summary of a systematic review. November 2016. www.supportsummaries.org

 

Keywords

 

evidence-informed health policy, evidence-based, systematic review, health systems research, health care, low and middle-income countries, developing countries, external supervision, quality of health care, compliance with quality standards, hospital quality of care

 

This summary was prepared with additional support from:

Pontificia Universidad Catolica de Chile.



Comments