The investigation and analysis of critical incidents and adverse events in healthcare

M Woloshynowych,^1* S Rogers,² S Taylor-Adams³ and C Vincent¹

¹ Clinical Safety Research Unit, Imperial College London, UK
² Department of Primary Care and Population Studies, University College London, UK
³ National Patient Safety Agency, London, UK

* Corresponding author

In other high-risk industries, learning from accidents and near misses is a long-established practice, in fact it is a cornerstone of safety analysis and improvement. In contrast, learning within healthcare has often been fragmentary and uncertain. In the last 10 years, however, sufficient work has accumulated within healthcare to warrant a review of methods of investigation and analysis, supplemented by a parallel overview of methods of investigation and analysis in other settings.

Objectives of the review

The objectives of the review were:

to carry out a review of published and unpublished work on the analysis of methods of accident investigation in high-risk industries and to provide a sound conceptual and practical foundation for the review of healthcare methods
to carry out a review of published and unpublished work on the analysis of critical incidents in healthcare
to develop guidelines for the analysis of critical incidents in healthcare for the hospital sector, mental health and primary care
to pilot the three sets of guidelines.

Review of techniques of accident analysis in high-risk industries

The diversity of techniques used in other industries greatly impressed us, as did the clarity with which they were presented and the power and conceptual development of some of the methods. A search of relevant databases, websites and specialist literature yielded 19 accident investigation and analysis techniques. Of these, 12 were selected and reviewed in detail. All had some strong points, although the approaches varied in comprehensiveness, theoretical adequacy, use of resources and the extent to which they were used and accepted. Some techniques stood out as being of particular value. For instance, MORT (Management Oversight and Risk Tree), if carried out completely, is an extremely comprehensive technique examining an accident from several perspectives using a toolbox of techniques. Many of these techniques provide useful methods of solving specific accident investigation or analysis problems. For example, barrier analysis is an exceptionally quick and useful approach to identifying where and how to implement specific types of defences and barriers within an organisation.

Review of studies of healthcare approaches

Initial searches on electronic and other databases identified 1950 potentially relevant papers. After screening of abstracts, 562 papers were obtained for further review. After further screening, 138 papers were identified for formal appraisal and a further 114 were designated as providing potentially useful background information.

A formal appraisal instrument was designed, piloted and modified until acceptable reliability was achieved. From the 138 papers, six techniques were identified as representing clearly definable approaches to incident investigation and analysis. We excluded from formal appraisal those techniques which had been used in less than five peer-reviewed published studies. All relevant papers, to a maximum of ten, were reviewed for each of the six techniques: Australian Incident Monitoring System (AIMS), the Critical Incident Technique, Significant Event Auditing (SEA), Root Cause Analysis (RCA), Organisational Accident Causation Model (OACM) and Comparison with Standards approach.

All techniques had the potential to be applied in any specialty or discipline related to healthcare. Although a few studies looked solely at death as an outcome, most used a variety of outcomes including near misses. Most techniques used interviewing and primary document review to investigate incidents. All techniques included papers which identified clinical issues and some attempt to assess underlying errors, causes and contributory factors. However, the extent and sophistication of the various attempts varied widely. Only one-third of papers referred to an established model of accident causation. In most studies examined there was little or no information on the training of investigators, how the data were extracted or any information on quality assurance for data collection and analysis. There was some variation in the level of expertise and training required, but to undertake the investigation to an acceptable depth all required some expertise. In most papers there was little or no discussion of implementation of any changes as a result of the investigations. One-quarter of publications gave some description of the implementation of changes, although few addressed evaluation of changes.

The development and piloting of a guide for the investigation and analysis of critical incidents and adverse events in healthcare

The review of methods of accident investigation in high-risk industries showed that there are a number of potentially useful techniques that could be used in healthcare. Review of techniques used in healthcare revealed two of particular interest and potential, RCA and OACM, but there were also methodological developments in other approaches that might be transferable (e.g. group-based approaches in SEA, taxonomies from the monitoring studies, links to implementation in audit and peer review approaches). Our learning from these techniques underpins the guide that appears in this publication. For three specialities, acute care, mental health and primary care, a research group was set up to test and pilot a draft version of the guide. Changes were then made following their experiences, comments and discussions. The resulting guide is included in Chapter 6 of the report, with case examples in the corresponding appendix.

The future of incident investigation in healthcare

The principal recommendations were as follows.

Defining the technique and providing manuals and guidelines

Manuals and descriptions of the methods of investigation and analysis need to be developed. Researchers need to provide much more detail on the purpose of the technique, its context of use and the process of investigation.

Resources and the need for training

High-risk industries recognise that accident investigation is a specialist and complex task, which requires substantial investment in training dedicated accident investigators. Healthcare professionals engaged in investigations also need adequate training and experience. Local teams need sufficient time to enable them to produce a thorough report with serious attention to implementing changes.

Implementation of changes

Both researchers and investigation teams need to give more attention to recommendations for change and implementation of changes. Research studies cannot always consider the whole cycle of investigation, analysis, implementation and evaluation, but as the techniques develop more attention should be given to linking findings directly to future prevention.

Integration of techniques

The range of effective approaches available in high-risk industries suggests that investigators of clinical incidents should think in terms of a ‘toolbox’ of approaches, where specific techniques would be used for different purposes and at different stages of an investigation.

Conclusion

Our reviews demonstrate that, while much valuable work has been accomplished, there is considerable potential for further development of techniques, the utilisation of a wider range of techniques and a need for validation and evaluation of existing methods, which would make incident investigation more versatile and use limited resources more effectively.

Future research

Further exploration of techniques used in high-risk industries, with interviews and observation of actual investigations, should prove valuable. Existing healthcare techniques would benefit from formal evaluation of their outcomes and effectiveness. Studies should examine depth of investigation and analysis, adequacy and feasibility of recommendations and cost-effectiveness. Examining implementation of recommendations is a key issue.

Publication

Woloshynowych M, Rogers S, Taylor-Adams S, Vincent C. The investigation and analysis of critical incidents and adverse events in healthcare. Health Technol Assess 2005;9(19).

NHS R&D HTA Programme

The research findings from the NHS R&D Health Technology Assessment (HTA) Programme directly influence key decision-making bodies such as the National Institute for Health and Clinical Excellence (NICE) and the National Screening Committee (NSC) who rely on HTA outputs to help raise standards of care. HTA findings also help to improve the quality of the service in the NHS indirectly in that they form a key component of the ‘National Knowledge Service’ that is being developed to improve the evidence of clinical practice throughout the NHS.

The HTA Programme was set up in 1993. Its role is to ensure that high-quality research information on the costs, effectiveness and broader impact of health technologies is produced in the most efficient way for those who use, manage and provide care in the NHS. ‘Health technologies’ are broadly defined to include all interventions used to promote health, prevent and treat disease, and improve rehabilitation and long-term care, rather than settings of care.

The HTA programme commissions research only on topics where it has identified key gaps in the evidence needed by the NHS. Suggestions for topics are actively sought from people working in the NHS, the public, consumer groups and professional bodies such as Royal Colleges and NHS Trusts.

Research suggestions are carefully considered by panels of independent experts (including consumers) whose advice results in a ranked list of recommended research priorities. The HTA Programme then commissions the research team best suited to undertake the work, in the manner most appropriate to find the relevant answers. Some projects may take only months, others need several years to answer the research questions adequately. They may involve synthesising existing evidence or designing a trial to produce new evidence where none currently exists.

Additionally, through its Technology Assessment Report (TAR) call-off contract, the HTA Programme is able to commission bespoke reports, principally for NICE, but also for other policy customers, such as a National Clinical Director. TARs bring together evidence on key aspects of the use of specific technologies and usually have to be completed within a limited time period.

Criteria for inclusion in the HTA monograph series

Reports are published in the HTA monograph series if (1) they have resulted from work commissioned for the HTA Programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees and editors.

Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search, appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the replication of the review by others.

The research reported in this monograph was commissioned by the HTA Programme as project number 98/28/05. As funder, by devising a commissioning brief, the HTA Programme specified the research question and study design. The authors have been wholly responsible for all data collection, analysis and interpretation and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

The views expressed in this publication are those of the authors and not necessarily those of the HTA Programme or the Department of Health.

Editor-in-Chief: Professor Tom Walley
Series Editors: Dr Peter Davidson, Professor John Gabbay, Dr Chris Hyde, Dr Ruairidh Milne, Dr Rob Riemsma and Dr Ken Stein
Managing Editors: Sally Bailey and Caroline Ciupek

HTA Home Page | Details page for this publication | Publications listing | Publications search