S Goodacre,* P Thokala, C Carroll, JW Stevens, J Leaviss, M Al Khalaf, P Collinson, F Morris, P Evans and J Wang

School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK

*Corresponding author

Background

Chest pain due to suspected acute coronary syndrome (ACS) is responsible for a large and increasing number of hospital attendances and admissions. Current standard practice involves using troponin I (TnI) or troponin T (TnT) to diagnose myocardial infarction (MI), measured on admission and at least 10 hours after symptom onset to allow for the limited early sensitivity of troponin. The development of high-sensitivity troponin assays and alternative biomarkers has raised the possibility of early diagnosis of MI with reduced hospital length of stay and health-care costs, but with potentially higher rates of initial misdiagnosis. Determining the optimal strategy for MI diagnosis involves weighing the costs and benefits of accurate diagnosis.

Once MI has been ruled out, the risk of future adverse events can be estimated using biomarkers of ischaemia or inflammation, exercise electrocardiography (ECG) or computed tomographic coronary angiography (CTCA), with antithrombotic treatment or coronary intervention being used to reduce the risk of adverse outcome in those with positive tests. Determining the optimal strategy involves weighing the benefits of reducing adverse events against costs of additional investigation and treatment.

Objectives

We undertook systematic reviews and meta-analysis to estimate:

1. the diagnostic accuracy of early biomarkers (including troponin) for MI
2. the prognostic accuracy of biomarkers for predicting major adverse cardiac adverse events (MACEs) in troponin-negative patients
3. the diagnostic accuracy of CTCA and exercise ECG for coronary artery disease (CAD) in patients with suspected ACS
4. the prognostic accuracy of CTCA and exercise ECG for predicting MACE in patients with suspected ACS.

We then developed an economic model to:

1. estimate the cost-effectiveness [measured as the cost per quality-adjusted life-year (QALY) gained by each strategy] of using early biomarker strategies to diagnose MI
2. estimate the cost-effectiveness of using biomarkers, CTCA and exercise ECG to risk-stratify patients with troponin-negative suspected ACS
3. identify the optimal strategies for diagnosing MI and investigating troponin-negative patients in the NHS, defined as the most cost-effective strategy at the National Institute for Health and Clinical Excellence threshold for willingness to pay per QALY gained
4. identify the critical areas of uncertainty in the management of suspected ACS, where future primary research would produce the most benefit.

Methods

The systematic reviews and meta-analysis were undertaken in accordance with the guidelines published by the Centre for Reviews and Dissemination for undertaking systematic reviews and the Cochrane Diagnostic Test Accuracy Working Group on the meta-analysis of diagnostic tests. Separate searches were undertaken for (1) biomarkers and (2) CTCA or exercise ECG. We searched electronic databases up to November 2010, reviewed citation lists and contacted experts to identify diagnostic and prognostic cohort studies comparing a relevant index test (biomarker, CTCA or exercise ECG) to the appropriate reference standard [MI according the universal definition, CAD on invasive coronary angiography (ICA) or MACE] in patients presenting with suspected ACS. The quality of diagnostic studies was assessed according to the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool. The quality of prognostic studies was assessed using an adapted version of the framework described by Altman (Systematic reviews of evaluations of prognostic variables. BMJ 2001;323:224–8). Meta-analysis was conducted using Bayesian Markov chain Monte Carlo simulation.

We developed a decision-analysis model to evaluate the cost-effectiveness of using (1) early biomarker strategies to diagnose MI before a 10-hour troponin test and (2) biomarkers, CTCA or exercise tolerance test to risk-stratify patients with a negative troponin. The model applied diagnostic strategies to a hypothetical cohort of patients with suspected ACS to determine the costs and outcomes associated with each strategy. The model involved two phases:

1. In the diagnostic phase, early biomarker strategies (involving troponin alone or in combination with sensitive early biomarkers) were compared with the most effective and expensive strategy of 10-hour troponin testing and the least effective and cheapest strategy of no testing or treatment.
2. In the prognostic phase, biomarkers and other investigations (CTCA and exercise ECG) were compared with a no-testing strategy and an ICA for all strategy in patients with negative troponin. The potential benefit of additional biomarkers, CTCA or exercise ECG was assumed to relate to identifying which patients have a higher risk of MACEs, which could be reduced by investigation and intervention.

We tested the diagnostic model in three different scenarios, depending on the availability of doctors to act on 10-hour troponin results, and two different populations, depending on whether the patient had known CAD or not. Estimates of diagnostic and prognostic accuracy were derived from the literature review. Our estimate of the benefit of detecting and treating MI, or of predicting adverse events, was derived from a recent observational study. Cost and utility estimates were derived from previous studies and routine data sources. The economic model was developed using SIMUL8 software (SIMUL8 Corporation, Boston, MA, USA), taking a health-service perspective and a lifetime horizon with mean life expectancy based on UK interim life tables. Deterministic and probabilistic analyses were undertaken.

Results

The biomarker review identified 2865 citations, from which we selected 40 diagnostic and 44 prognostic studies that met our inclusion criteria and had data that could be extracted. Studies of presentation TnI (n = 21) and TnT (n = 11) evaluated a variety of different assays using different thresholds for positivity. Studies with similar assays and thresholds were grouped together for meta-analysis. The summary estimates of sensitivity and specificity of TnI for MI were 77% (95% predictive interval 29–96%) and 93% (95% predictive interval 46–100%), respectively, when the 99th percentile was used and 82% (95% predictive interval 40–97%) and 93% (95% predictive interval 74–98%) when the 10% coefficient of variation (CV) was used. The corresponding results for TnT were 80% (95% predictive interval 33–97%) and 91% (95% predictive interval 53–99%) when the 99th percentile was used and 74% (95% predictive interval 34–94%) and 96% (95% predictive interval 76–99%) when the 10% CV was used. Meta-analysis was also undertaken for three high-sensitivity assays using the 99th percentile threshold. The sensitivity and specificity were 96% (95% predictive interval 27–100%) and 72% (95% predictive interval 3–96%), respectively, for the Roche high-sensitivity TnT assay (Roche Diagnostics, Basel, Switzerland), 86% (95% predictive interval 22–96%) and 89% (95% predictive interval 40–97%), respectively, for the ADVIA Centaur Ultra troponin I assay (Siemens Healthcare, Erlangen, Germany), and 83% (95% predictive interval 58–95%) and 95% (95% predictive interval 67–100%) for the Abbot Architect troponin I assay (Abbott Laboratories, IL, USA).

We selected 17 studies of heart-type fatty acid-binding protein (H-FABP) and analysed quantitative (n = 8) and qualitative (n = 9) assays separately. The summary estimates of sensitivity and specificity for MI were 81% (95% predictive interval 50–95%) and 80% (26–98%), respectively, for the quantitative assays and 68% (11–97%) and 92% (20–100%), respectively, for the qualitative assays. Meta-analysis of four studies of ischaemia-modified albumin yielded summary estimates of 77% (19–98%) for sensitivity and 39% (2–95%) for specificity. Meta-analysis of 17 studies of myoglobin yielded summary estimates of 62% (35–83%) for sensitivity and 83% (35–98%) for specificity. We also identified 10 studies of nine other biomarkers with insufficient numbers of studies for meaningful meta-analysis. Another nine studies reported combinations of biomarkers with troponin compared with troponin alone, showing that combining these biomarkers with troponin increases sensitivity for MI at the expense of specificity.

The 44 prognostic biomarker studies reported associations between a number of different biomarkers and risk of MACEs. Some 26 studies undertook multivariate analysis, showing that B-type natriuretic peptide, N-terminal-pro-BNP, myeloperoxidase and H-FABP can provide additional prognostic value beyond troponin, whereas 11 studies analysed troponin-negative patients separately to show that C-reactive protein, pregnancy-associated plasma protein-A and H-FABP can predict MACEs in these patients.

The CTCA and exercise ECG review identified 2342 citations, from which we selected 15 CTCA papers (eight diagnostic and seven prognostic) and 13 exercise ECG papers (all prognostic) that fulfilled our inclusion criteria. The diagnostic studies of CTCA were relatively small (n= 31–113) and mostly used 50% stenosis to define obstructive CAD in both the index and reference standard test. Summary estimates of sensitivity and specificity were 94% (95% predictive interval 61–99%) and 87% (95% predictive interval 16–100%), respectively. The prognostic studies of CTCA were generally larger (n = 30–588) than the diagnostic studies, but MACE rates were generally low. Definitions of MACEs varied and some studies did not report outcomes for those with positive CTCA. Meta-analysis of the five studies with analysable data showed a relative risk (RR) for MACEs of 3.1 (95% predictive interval 0.3–18.7) for positive and intermediate scan compared with negative scan and 5.8 (95% predictive interval 0.6–24.5%) for positive compared with intermediate or negative scan.

There were no diagnostic studies of exercise ECG. The prognostic studies ranged from n = 28 to n = 1000 with varying definitions of MACEs. Meta-analysis showed a RR for MACEs of 8.4 (95% predictive interval 3.1–17.3) for positive and inconclusive tests compared with negative test and 8.0 (95% predictive interval 2.3–22.7) for positive test compared with inconclusive or negative test.

In the economic analysis the main diagnostic model showed that the optimal strategy in all but one scenario was measurement of high-sensitivity troponin at presentation, with a 10-hour troponin test if positive and discharge home if negative. This strategy was the most effective strategy, with an incremental cost-effectiveness ratio (ICER) of less than the £20,000–30,000/QALY threshold (ICER £7487–17,191/QALY). The exception was the scenario involving patients without known CAD and doctor available on demand to discharge the patient, using the £30,000/QALY threshold, where a strategy of measuring a 10-hour troponin level in all patients was more effective and had an ICER of £27,546/QALY. Sensitivity analysis suggested that if presentation high-sensitivity troponin had lower sensitivity than the baseline estimate (86% as opposed to 96%) then the 10-hour troponin strategy would be the most cost-effective in half the scenarios using the £30,000/QALY threshold and in one scenario using the £20,000/QALY threshold. Sensitivity analysis also suggested that, if included, a strategy of measuring high-sensitivity troponin at presentation and 3 hours later would be optimal and the 10-hour strategy would then have an ICER of > £100,000/QALY in all scenarios.

A secondary analysis using data from individual studies comparing the combination of troponin and another biomarker to troponin alone showed that the addition of H-FABP, copeptin or myoglobin appeared to be cost-effective, with ICERs of < £20,000–30,000/QALY, whereas the addition of ischaemia-modified albumin to troponin was not cost-effective. However, the troponin assays used for comparison in this analysis were not all high-sensitivity assays and had generally lower sensitivity than troponin in the main analysis.

The main prognostic model showed that CTCA appeared to be the most cost-effective strategy, with an ICER of £11,041/QALY. ICA for all was more effective but with an ICER of £219,532. Probabilistic analysis showed that CTCA was most likely to be cost-effective at the £20,000–30,000/QALY threshold. However, these findings were dependent on the estimated rate of MACEs. When an alternative data source with a lower MACE rate was used the no-testing strategy was optimal, CTCA had a high ICER (£262,061/QALY) and ICA was dominated, with higher costs and worse outcomes than no testing. A threshold analysis revealed that CTCA was likely to be cost-effective if the combined risk of death and non-fatal MI within the time period assumed to be influenced by initial diagnostic testing exceeded 2% (£30,000/QALY threshold) or 2.9% (£20,000/QALY threshold).

Conclusions

Main findings

The sensitivity of troponin at presentation is around 70–80% depending on the assay used. High-sensitivity assays have a sensitivity at presentation of around 80–95%, but with some apparent loss of specificity. Studies are subject to much heterogeneity and estimates are consequently surrounded by substantial uncertainty. Compared with the ‘gold standard’ of a 10-hour troponin test, even a high-sensitivity presentation troponin test will miss a significant minority of patients with MI. However, economic analysis suggests that the additional costs that are likely to be incurred by measuring a 10-hour troponin level, compared with a presentation high-sensitivity troponin level, are unlikely to represent a cost-effective use of NHS resources in most of the scenarios tested.

There is some evidence from individual studies that H-FABP, copeptin and myoglobin may be used alongside troponin to increase early sensitivity in a cost-effective manner. However, these findings need to be confirmed in further studies comparing biomarker combinations to high-sensitivity troponin assays.

The limited diagnostic evidence available for CTCA suggests that diagnostic accuracy for CAD in patients with suspected ACS is similar to that previously estimated for patients with stable symptoms. There are no diagnostic studies to estimate the accuracy of exercise ECG for CAD in patients with suspected ACS. Prognostic studies of both CTCA and exercise ECG are limited by low MACE rates and the use of process outcomes in unblinded studies, but provide weak evidence that either investigation can be used to predict MACEs in patients with suspected ACS.

Economic evaluation of using biomarkers, exercise ECG, CTCA or ICA in troponin-negative patients with suspected ACS suggests that CTCA may be the most cost-effective strategy, but that cost-effectiveness (and essentially the effectiveness) of CTCA is dependent on the expected risk of a MACE. If the combined risk of death and MI is < 2% then CTCA is unlikely to be cost-effective. Furthermore, weaknesses in the source data used in the model substantially limit the reliability of conclusions.

Implications for practice

Hospital admission for 10-hour troponin testing is unlikely to be cost-effective compared with high-sensitivity troponin at presentation unless rapid decision-making and discharge is possible. Removing the recommendation for 10-hour troponin testing from guidance could reduce the need for hospital admission among patients awaiting delayed troponin testing. However, the use of high-sensitivity troponin testing has the potential to increase the incidence of MI diagnosis and thus demand for cardiology services. There is currently insufficient evidence to support routine use of alternative biomarkers alongside troponin or routine investigation with exercise ECG or CTCA in troponin-negative patients.

Research recommendations

Evaluation of:

1. the diagnostic accuracy of troponin and alternative biomarkers at presentation, 3 hours and 10 hours, and of the prognostic accuracy of CTCA in a large multicentre cohort study of patients presenting with suspected ACS
2. the effect of using high-sensitivity troponin in the diagnostic assessment of suspected ACS, compared with standard troponin, on event rates and health-care costs in a clinical trial and economic evaluation
3. the effect of early CTCA for all patients with troponin-negative ACS compared with current standard practice, on event rates and health-care costs in a clinical trial and economic evaluation.

Funding

Funding for this study was provided by the Health Technology Assessment programme of the National Institute for Health Research.

Publication

Goodacre S, Thokala P, Carroll C, Stevens JW, Leaviss J, Al Khalaf M, et al. Systematic review, meta-analysis and economic modelling of diagnostic strategies for suspected acute coronary syndrome. Health Technol Assess 2013;17(01).

NIHR Health Technology Assessment programme

The Health Technology Assessment (HTA) programme, part of the National Institute for Health Research (NIHR), was set up in 1993. It produces high-quality research information on the effectiveness, costs and broader impact of health technologies for those who use, manage and provide care in the NHS. ‘Health technologies’ are broadly defined as all interventions used to promote health, prevent and treat disease, and improve rehabilitation and long-term care.

The research findings from the HTA programme directly influence decision-making bodies such as the National Institute for Health and Clinical Excellence (NICE) and the National Screening Committee (NSC). HTA findings also help to improve the quality of clinical practice in the NHS indirectly in that they form a key component of the ‘National Knowledge Service’.

The HTA programme is needs led in that it fills gaps in the evidence needed by the NHS. There are three routes to the start of projects.

First is the commissioned route. Suggestions for research are actively sought from people working in the NHS, from the public and consumer groups and from professional bodies such as royal colleges and NHS trusts. These suggestions are carefully prioritised by panels of independent experts (including NHS service users). The HTA programme then commissions the research by competitive tender.

Second, the HTA programme provides grants for clinical trials for researchers who identify research questions. These are assessed for importance to patients and the NHS, and scientific rigour.

Third, through its Technology Assessment Report (TAR) call-off contract, the HTA programme commissions bespoke reports, principally for NICE, but also for other policy-makers. TARs bring together evidence on the value of specific technologies.

Some HTA research projects, including TARs, may take only months, others need several years. They can cost from as little as £40,000 to over £1 million, and may involve synthesising existing evidence, undertaking a trial, or other research collecting new data to answer a research problem.

The final reports from HTA projects are peer reviewed by a number of independent expert referees before publication in the widely read journal series Health Technology Assessment.

Criteria for inclusion in the HTA journal series

Reports are published in the HTA journal series if (1) they have resulted from work for the HTA programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees and editors.

Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search, appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the replication of the review by others.

The research reported in this issue of the journal was funded by the HTA programme as project number 09/22/21. The contractual start date was in July 2010. The draft report began editorial review in January 2012 and was accepted for publication in June 2012. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.

Editor-in-Chief: Professor Tom Walley CBE
Series Editors: Dr Martin Ashton-Key, Professor Aileen Clarke, Dr Peter Davidson, Ms Tara Lamont, Dr Tom Marshall, Professor William McGuire, Professor Jane Norman, Professor John Powell, Professor James Raftery, Dr Rob Riemsma, Professor Helen Roberts, Professor Helen Snooks and Professor Ken Stein