AK Ewer,^1,2* AT Furmston,³ LJ Middleton,³ JJ Deeks,⁴ JP Daniels,^1,3 HM Pattison,⁵ R Powell,⁵ TE Roberts,⁶ P Barton,⁶ P Auguste,⁶ A Bhoyar,⁷ S Thangaratinam,^1,2 AM Tonks,⁸ P Satodia,⁹ S Deshpande,¹⁰ B Kumararatne,¹¹ S Sivakumar,¹² R Mupanemunda¹³ and KS Khan¹⁴

¹University of Birmingham, School of Clinical and Experimental Medicine, Birmingham, UK
²Birmingham Women’s Healthcare NHS Foundation Trust, Birmingham, UK
³University of Birmingham, Birmingham Clinical Trials Unit, Birmingham, UK
⁴University of Birmingham, Public Health, Epidemiology and Biostatistics, Birmingham, UK
⁵Aston University, School of Life and Health Sciences, Birmingham, UK
⁶University of Birmingham, Health Economics Unit, School of Health and Population Sciences, Birmingham, UK
⁷Birmingham Children’s Hospital NHS Foundation Trust, Birmingham, UK
⁸West Midlands Perinatal Institute, Birmingham, UK
⁹University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
¹⁰The Shrewsbury and Telford Hospital NHS Trust, Shrewsbury, UK
¹¹The Royal Wolverhampton Hospitals NHS Trust, Wolverhampton, UK
¹²Sandwell and West Birmingham Hospitals NHS Trust, Birmingham, UK
¹³Heart of England NHS Foundation Trust, Birmingham, UK
¹⁴Barts and the London School of Medicine and Dentistry, London, UK

*Corresponding author

Background

Congenital heart defects (CHDs) are the most common group of congenital malformations and one of the leading causes of infant death in the developed world. Early detection of major CHDs – which cause death or require invasive intervention before 1 year of age – may improve outcome. Current routine screening for CHDs relies on a mid-trimester anomaly ultrasound scan in pregnant women, involving imaging of the heart chambers, and a postnatal clinical examination involving assessment of the cardiovascular system. Both of these have a relatively low detection rate and a number of babies are discharged from hospital before a CHD is diagnosed. A proportion of these may die or present in such a poor clinical condition that the outcome, despite treatment, is compromised.

This report assesses the accuracy, acceptability to both parents and clinical staff, and the cost-effectiveness of an alternative approach, based on pulse oximetry screening, to determine the value of this method in diagnosing those CHD lesions that are potentially life-threatening.

Objectives

This health technology assessment completed three distinct pieces of work, the objectives of which were to determine:

• the accuracy of pulse oximetry against the composite reference standard of echocardiography, clinical follow-up and interrogation of regional and national clinical databases, and to determine the added value of pulse oximetry over routine antenatal ultrasound screening
• the acceptability of pulse oximetry testing to parents and health-care staff
• the cost-effectiveness of pulse oximetry testing compared with existing strategies for CHD screening.

Methods

A test accuracy study was performed in six large maternity units with delayed verification in test negatives. The index test of pulse oximetry testing was performed in 20,055 eligible newborns prior to discharge from hospital. Those not achieving predetermined oxygen saturation thresholds underwent the reference standard of echocardiography. All other infants were followed up to 12 months of age through the interrogation of regional and national congenital anomaly and cardiac registries and clinical follow-up. The study compared the accuracy of the index test in detecting major CHDs subdivided into critical (causing death or requiring invasive intervention before 28 days) and serious (causing death or requiring invasive intervention between 1 and 12 months of age).

Acceptability of testing to participants was evaluated through a structured questionnaire completed after testing and again at 1 year in a subsample. The characteristics of those who declined to take part in the study when first approached were also analysed. Acceptability of testing to clinical staff was evaluated through focus groups and e-mail surveys with those who had taken part in the study.

For the economic evaluation, resource usage data were collected alongside the test accuracy study to establish the cost of pulse oximetry testing. A decision-analytic model was constructed to estimate the cost-effectiveness of various screening and prevention strategies, using an NHS perspective and an outcome of cost per case of timely diagnosis of major CHDs.

Results

Main findings of test accuracy study

There were 53 cases of major CHDs (24 critical and 29 serious) within the study cohort of 20,055 babies. Of those with an abnormal result following pulse oximetry testing, 26/195 [13.33%; 95% confidence interval (CI) 8.9% to 18.92%] had major CHDs (18 critical and 8 serious). Of the babies who had a normal pulse oximetry result, 27/19,860 (0.14%; 95% CI 0.09% to 0.20%) had major CHDs (6 critical and 21 serious).

For the full cohort, pulse oximetry had a sensitivity of 75.0% (95% CI 53.3% to 90.2%) for critical cases and 49.1% (95% CI 35.1% to 63.2%) for critical plus serious cases combined. For the cohort in which CHDs had not been suspected following antenatal ultrasound, pulse oximetry had a sensitivity of 58.3% (95% CI 27.7% to 84.8%) for critical cases (12 babies) and 28.6% (95% CI 14.6% to 46.3%) for critical plus serious cases combined (35 babies). One in 119 babies (0.84%) without serious or critical CHDs had a false-positive (FP) result (specificity 99.16%, 95% CI 99.02% to 99.28%). In addition, in the FP cohort, 6/169 (3.5%) of babies had CHDs defined as significant but not major, and 40/169 (24%) had a respiratory or infective illness requiring hospital treatment. Thus, in the test-positive cohort, in total, 72/195 (37%) had a condition requiring medical intervention.

Main findings of acceptability studies

Across all parts of the study, parent and staff participants were predominantly satisfied with pulse oximetry screening, perceiving it to be an important and valuable test to detect unwell babies. There was no evidence that mothers given FP results were more anxious after taking part in the screening processes than those given true-negative (TN) results, although the former were less satisfied with the test and gave higher depression scores (a small, but statistically significant difference). In multivariate analyses, higher anxiety and depression was predicted by lower optimism, lower overall satisfaction and ethnicity (white British/Irish participants being less anxious). Satisfaction with screening was predicted by higher perception of the treatment’s ability to control heart disease, comprehensibility of heart disease, and lower stress, anxiety and depression. White British/Irish participants were more satisfied than those of other ethnicities. Indian mothers were less satisfied overall with screening and Pakistani mothers were more stressed during screening than white British/Irish mothers. Communication problems were indicated as a cause of worry by participants, and staff identified a need for further training in communicating with parents about the study and for giving results, especially where a positive result is found. Malfunctioning equipment could increase anxiety for parents and staff alike.

Despite creating an additional workload in testing babies, the test was seen as reassuring for staff, and positively impacted on the roles of those caring for ill babies as they could be treated while in a less critical condition.

Main findings of health economics

The pulse oximetry test took on average 6.9 minutes (median 5 minutes, range 1 to 30 minutes) to be completed. Taking into account equipment costs, the procedure is estimated to cost £6.24 per test. The additional cost of including pulse oximetry as an adjunct to the current practice of clinical examination of the newborn was estimated to be approximately £24,900 per additional case of timely diagnosis. This estimate was shown to be robust in an extensive sensitivity analysis.

Conclusions

Implications for health care

Pulse oximetry is a safe, simple, non-invasive, feasible and reasonably accurate test that is acceptable to parents and clinical staff, and has a sensitivity that appears to be superior to that of antenatal screening and clinical examination. The use of both pre- and postductal saturations compared with postductal alone appears to be advantageous and, in practice, does not take significantly longer to perform.

Pulse oximetry adds value to existing screening procedures and is likely to identify cases of critical CHD that would otherwise go undetected. The detection of other pathologies, such as significant CHDs and respiratory and infective illnesses, is an additional advantage.

Pulse oximetry as an adjunct to clinical examination is twice as costly, but will detect more babies with CHDs.

Recommendations for research

Pulse oximetry improves detection rates of critical CHDs. The majority of critical cases missed by pulse oximetry (and by other screening methods) are associated with obstruction of the aortic arch as these conditions are often not associated with hypoxaemia. Further investigation of other oximetry techniques, such as perfusion index, may enhance the detection rates for these lesions.

Further research should be conducted with mothers of different ethnicities to gain a greater understanding of factors limiting participation and satisfaction with testing. Research should also be conducted with mothers given FP results at the time of discharge from hospital to verify whether they experience heightened anxiety at this point.

Funding

The National Institute for Health Research Health Technology programme.

Publication

Ewer AK, Furmston AT, Middleton LJ, Deeks JJ, Daniels JP, Pattison HM, et al. Pulse oximetry as a screening test for congenital heart defects in newborn infants: a test accuracy study with evaluation of acceptability and cost-effectiveness.. Health Technol Assess 2012;16(2).

NIHR Health Technology Assessment programme

The Health Technology Assessment (HTA) programme, part of the National Institute for Health Research (NIHR), was set up in 1993. It produces high-quality research information on the effectiveness, costs and broader impact of health technologies for those who use, manage and provide care in the NHS. ‘Health technologies’ are broadly defined as all interventions used to promote health, prevent and treat disease, and improve rehabilitation and long-term care.

The research findings from the HTA programme directly influence decision-making bodies such as the National Institute for Health and Clinical Excellence (NICE) and the National Screening Committee (NSC). HTA findings also help to improve the quality of clinical practice in the NHS indirectly in that they form a key component of the ‘National Knowledge Service’.

The HTA programme is needs led in that it fills gaps in the evidence needed by the NHS. There are three routes to the start of projects.

First is the commissioned route. Suggestions for research are actively sought from people working in the NHS, from the public and consumer groups and from professional bodies such as royal colleges and NHS trusts. These suggestions are carefully prioritised by panels of independent experts (including NHS service users). The HTA programme then commissions the research by competitive tender.

Second, the HTA programme provides grants for clinical trials for researchers who identify research questions. These are assessed for importance to patients and the NHS, and scientific rigour.

Third, through its Technology Assessment Report (TAR) call-off contract, the HTA programme commissions bespoke reports, principally for NICE, but also for other policy-makers. TARs bring together evidence on the value of specific technologies.

Some HTA research projects, including TARs, may take only months, others need several years. They can cost from as little as £40,000 to over £1 million, and may involve synthesising existing evidence, undertaking a trial, or other research collecting new data to answer a research problem.

The final reports from HTA projects are peer reviewed by a number of independent expert referees before publication in the widely read journal series Health Technology Assessment.

Criteria for inclusion in the HTA journal series

Reports are published in the HTA journal series if (1) they have resulted from work for the HTA programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees and editors.

Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search, appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the replication of the review by others.

The research reported in this issue of the journal was commissioned by the HTA programme as project number 06/06/03. The contractual start date was in July 2007. The draft report began editorial review in October 2010 and was accepted for publication in June 2011. As the funder, by devising a commissioning brief, the HTA programme specified the research question and study design. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.

Editor-in-Chief: Professor Tom Walley CBE
Series Editors: Dr Martin Ashton-Key, Professor Aileen Clarke, Dr Tom Marshall, Professor John Powell, Dr Rob Riemsma and Professor Ken Stein
Associate Editor: Dr Peter Davidson