Executive summary of HTA journal title
Health Technol Assess 2011;15(30):1–156
Identification of risk factors by systematic review and development of risk-adjusted models for surgical site infection.
View/Download full text in Adobe Acrobat format (1.6MB)
View/Download this summary in Adobe Acrobat format (suitable for printing)
C Gibbons,1 J Bruce,2 J Carpenter,1 AP Wilson,3 J Wilson,4 A Pearson,5 DL Lamping,1† ZH Krukowski6 and BC Reeves7*
1London School of Hygiene and Tropical Medicine, London, UK
2University of Aberdeen, Aberdeen, UK
3University College London, London, UK
4Imperial College Healthcare NHS Trust, London, UK
5Health Protection Agency, London, UK
6Aberdeen Royal Infirmary, Aberdeen, UK
7University of Bristol, Bristol, UK
Background to the research
Surgical site infections (SSIs) are complications of surgery that cause significant postoperative morbidity. They are costly to health services and inconvenient, painful and potentially fatal to affected patients. Rates of SSI have been observed to vary widely by hospital and may be influenced by surgical management and other aspects of the quality of health care. SSI rate (SSI%) has been proposed as a potential indicator of the quality of care in the context of clinical governance and monitoring of the performance of NHS organisations against targets.
The risk of developing an SSI is likely to be influenced by the characteristics of patients, of operations and postoperative care. Therefore, the use of SSI as a performance indicator requires hospital-specific rates to be risk adjusted. This research sought to identify important risk factors for SSI in defined contexts, whether surgery specific or generic, and investigate the feasibility of risk-adjusting SSI%.
Aim and objectives of the research
The aim of the proposed research was to investigate methods for the risk adjustment of rates of SSI. We proposed to address the following specific objectives.
- To identify risk factors for SSI, criteria for the stratification of surgical procedures and evidence about the importance of postdischarge surveillance (PDS) from systematic reviews of the literature.
- To test whether or not ‘short-listed’ variables from the literature are risk factors in available SSI surveillance databases. To identify in univariable analyses other potential risk factors from available databases and to investigate interactions between risk factors.
- To develop models for making risk-adjusted comparisons between hospitals.
- To investigate modifications of the definition of SSI used by the Centers for Disease Control and Prevention (CDC) and the impact of modified definitions on the importance (use for prediction) of risk factors identified.
How the research was conducted
Reviews of the literature
Four systematic reviews of the literature were carried out. These reviews sought to identify:
- surgery-specific risk factors for SSI following joint replacement
- surgery-specific risk factors for SSI following large bowel surgery
- generic risk factors (relevant to many surgical procedures) for SSI that are not included in existing SSI risk indices
- risk factors for SSIs detected by following up patients after discharge.
Systematic searches were conducted on two biomedical databases, MEDLINE and EMBASE (1966–2004 and 1980–2004, respectively). Search strategies consisted of medical subject headings and free-text terms relating to surgical infection (surgical wound infection/SSI/postoperative infection), risk adjustment (risk assessment/factor/adjustment/stratification/modelling) and, where appropriate, the surgical area being reviewed or terms describing PDS. The review also used literature identified by a previous systematic review.
Agreement between definitions of surgical site infection
This part of the research used data collected by SSI surveillance of cardiac, thoracic, orthopaedic, general, obstetric, gynaecological, urological, maxillofacial, plastic and vascular surgical specialties in one UK hospital. The data, for 5804 surgical wounds in 4773 patients, allowed four SSI definitions to be applied: (1) ASEPSIS (Additional treatment, the presence of Serous discharge, Erythema, Purulent exudate, and Separation of the deep tissues, the Isolation of bacteria and the duration of inpatient Stay); (2) 1992 definition of the CDC; (3) a modified version of the 1992 CDC definition used for SSI surveillance in England; and (4) a definition based on pus. Patients were contacted by post or telephone 1–2 months after their operations to complete a PDS questionnaire designed to detect SSIs arising after discharge from hospital. SSIs identified by different definitions were tabulated and agreement between definitions was quantified.
Validation of definitions of surgical site infection
This part of the research used an updated version of the above data set from the same UK hospital, describing 11,124 wounds in 8691 patients. We constructed a set of clinical outcomes that wound infection would be expected to influence or cause: (1) clinical actions that were likely to reflect both mild (prescription of antibiotic) and severe infection (wound retreated); (2) patients’ views about whether or not there was a problem with the healing of their wounds; and (3) length of hospital stay, reflecting health service resource use. Modifications were made to SSI definitions to try to ensure that they were independent of the outcomes. We then developed logistic regression models to quantify the ability of alternative SSI definitions to predict the outcomes.
Surgical site infection risk modelling
This part of the research used data submitted to the UK Surgical Site Infection Surveillance Service [at the time, the Nosocomial Infection National Surveillance Scheme (NINSS)]. Hospitals taking part carried out surveillance of one or more of 12 categories of surgical procedure, e.g. large bowel surgery, coronary artery bypass graft (CABG) or hip replacement. In order for data to be included, there was a requirement for a hospital to carry out surveillance for at least 3 consecutive months. Hospitals submitted data about key risk factors for SSI [including the National Nosocomial Infections Surveillance (NNIS) risk index, demographic information about patients, and characteristics of the operation and wound] and information about SSIs that developed during the hospital stay. Univariable logistic regression analyses were initially carried out on the entire data set of 113,824 operations, stratifying by surgical procedure and then for each procedure separately. Multivariable risk models, with hospitals fitted as random effects, were then developed for each procedure. Effect modification of risk factors by hospital was investigated in multilevel models.
Reviews of the literature
The literature reviewed was found to be mainly of poor methodological quality, preventing quantitative summaries of the risk conferred by specific risk factors. The reviews of surgery-specific risk factors, other than those which make up the established risk indices, identified other factors associated with increasing risk of SSI. This has also been suggested for operations other than those which we reviewed. Some risk factors are unequivocally surgery specific, but others may apply to a range of procedures. The factor most commonly identified by the review of generic risk factors was duration of preoperative stay in hospital. The review of SSIs detected by PDS demonstrated that a significant proportion of SSIs develop after discharge and that the need to include PDS is an important consideration for procedures where the length of hospital stay is short or likely to vary over time or between institutions.
Agreement between definitions of surgical site infection
There was wide variation in the frequency of SSI identified using different definitions. Using existing CDC and ASEPSIS definitions of SSI (most and least sensitive definitions), over twice as many wounds were classified as infected by one definition only as were classified as infected by both. Different SSI definitions also classified different wounds as being infected, although some wounds were classified as infected by all definitions.
Validation of definitions of surgical site infection
Both ASEPSIS and CDC SSI definitions had a broadly similar ability to predict the chosen clinical outcomes; areas under receiver operating characteristic curves ranged from 0.75 to 0.88, except for prediction of prolonged hospital stay (0.64). These findings are paradoxical given the poor agreement between definitions in classifying individual wounds. There may be elements of each definition that are important in identifying the outcomes but which are not common to both, or the ability to predict the outcomes may depend on only a subset of features that are common to both. These possibilities suggest that there is an opportunity to produce a better definition by combining the elements from different definitions or by dropping redundant ones.
Surgical site infection risk modelling
Univariable models highlighted that components of existing risk indices should be modelled separately and that there was effect modification of risk factors by surgical procedure. The risk factors included in best-fit multivariable models varied by surgical procedure, as did the effects of risk factors included in the models. This conclusion applies to components of existing risk indices as well as to other factors considered in the analyses. Of the components in established risk indices, operative duration appeared to be an important risk factor for all operations, except for hip replacement. Wound class was included least often because some wound classes were not applicable to some surgical procedures or were combined because of small numbers. The American Association of Anesthesiologists class was a consistent risk factor for most surgery categories (except open reduction of fractures); its effect was uncertain for limb amputation and vascular surgery because of the small sample sizes available.
Age and gender were included in all models. The odds of SSI clearly increased with age for four surgery categories (CABG, hip and knee prostheses and open reduction of fracture), but not for four other surgery categories (large and small bowel, limb amputation and vascular surgery). The results were most varied for gender. Women had lower odds of SSI for knee prosthesis and open reduction of fracture, higher odds of SSI for CABG and similar odds of SSI for small and large bowel surgery, hip prosthesis and limb amputation. Preoperative duration of stay, an additional generic risk factor identified by the reviews, was associated with an increase in the risk of SSI for the four surgery categories with the largest number of data (hip and knee prosthesis, CABG and large bowel surgery).
The research literature does not allow a set of surgery-specific or generic risk factors to be defined. We believe that there is a need for high-quality research to develop a revised SSI definition that has satisfactory psychometric properties and which can be applied in everyday clinical settings and to the surveillance for SSI after discharge from hospital. Research to identify risk factors for SSI needs to be carried out to higher methodological standards, primarily by following established epidemiological principles.
Surgical site infection definitions vary between surveillance programmes and, because they are complex and difficult to apply, potentially between hospitals within programmes. Definitions that are different, some in apparently only minor ways, do not have good agreement. The most widely established definitions have a similar ability to predict outcomes influenced by SSI.
In surgery-specific multivariable risk-adjusted models, associations between components of the NNIS risk index and the odds of SSI varied both quantitatively and qualitatively for different surgical procedures; this finding also applied to other risk factors investigated. There was no evidence for effect modification of risk factors by hospital.
Surveillance programmes are important to inform clinical governance and the management of infection control over time. Performance estimates (data quality and SSI%) based on consistent surveillance methods for institutions and groupings within institutions should be disseminated locally for this purpose. Comparisons of performance estimates (SSI%) for institutions or countries should be regarded with caution; nevertheless, comparisons against a benchmark may prompt institutions to make changes to infection control practices that are associated with improved performance. Judgements about the quality of medical care provided by hospitals should not be based on these statistics alone by agencies responsibility for auditing performance. National surveillance systems should comply with a set of features designed to ensure their quality.
Future research should focus on devloping an SSI definition that has satisfactory psychometric properties, that can be applied in everyday clinical settings, includes PDS and is formulated to detect SSIs that are important to patients or health services.
Funding for this study was provided by the Health Technology Assessment programme of the National Institute for Health Research.
Gibbons C, Bruce J, Carpenter J, Wilson AP, Wilson J, Pearson A, et al. Identification of risk factors by systematic review and development of risk-adjusted models for surgical site infection. Health Technol Assess 2011;15(30).
NIHR Health Technology Assessment programme
The Health Technology Assessment (HTA) programme, part of the National Institute for Health Research (NIHR), was set up in 1993. It produces high-quality research information on the effectiveness, costs and broader impact of health technologies for those who use, manage and provide care in the NHS. ‘Health technologies’ are broadly defined as all interventions used to promote health, prevent and treat disease, and improve rehabilitation and long-term care.
The research findings from the HTA programme directly influence decision-making bodies such as the National Institute for Health and Clinical Excellence (NICE) and the National Screening Committee (NSC). HTA findings also help to improve the quality of clinical practice in the NHS indirectly in that they form a key component of the ‘National Knowledge Service’.
The HTA programme is needs led in that it fills gaps in the evidence needed by the NHS. There are three routes to the start of projects.
First is the commissioned route. Suggestions for research are actively sought from people working in the NHS, from the public and consumer groups and from professional bodies such as royal colleges and NHS trusts. These suggestions are carefully prioritised by panels of independent experts (including NHS service users). The HTA programme then commissions the research by competitive tender.
Second, the HTA programme provides grants for clinical trials for researchers who identify research questions. These are assessed for importance to patients and the NHS, and scientific rigour.
Third, through its Technology Assessment Report (TAR) call-off contract, the HTA programme commissions bespoke reports, principally for NICE, but also for other policy-makers. TARs bring together evidence on the value of specific technologies.
Some HTA research projects, including TARs, may take only months, others need several years. They can cost from as little as £40,000 to over £1 million, and may involve synthesising existing evidence, undertaking a trial, or other research collecting new data to answer a research problem.
The final reports from HTA projects are peer reviewed by a number of independent expert referees before publication in the widely read journal series Health Technology Assessment.
Criteria for inclusion in the HTA journal series
Reports are published in the HTA journal series if (1) they have resulted from work for the HTA programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees and editors.
Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search, appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the replication of the review by others.
The research reported in this issue of the journal was commissioned by the National Coordinating Centre for Research Methodology (NCCRM), and was formally transferred to the HTA programme in April 2007 under the newly established NIHR Methodology Panel. The HTA programme project number is 06/90/19. The contractual start date was in April 2003. The draft report began editorial review in October 2010 and was accepted for publication in March 2011. The commissioning brief was devised by the NCCRM who specified the research question and study design. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.
The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.
Editor-in-Chief: Professor Tom Walley CBE
Series Editors: Dr Martin Ashton-Key, Professor Aileen Clarke, Dr Tom Marshall, Professor John Powell, Dr Rob Riemsma and Professor Ken Stein
Associate Editor: Dr Peter Davidson
© 2011 Crown Copyright Top ^