Bayesian methods in health technology assessment: a review

DJ Spiegelhalter ^{1 *}
JP Myles ¹
DR Jones ²
KR Abrams²

¹ MRC Biostatistics Unit, Institute of Public Health, Cambridge, UK
² Department of Epidemiology and Public Health, University of Leicester, Leicester, UK

^* Corresponding author

Background

Bayesian methods may be defined as the explicit quantitative use of external evidence in the design, monitoring, analysis, interpretation and reporting of a health technology assessment. In outline, the methods involve formal combination through the use of Bayes's theorem of:

a prior distribution or belief about the value of a quantity of interest (for example, a treatment effect) based on evidence not derived from the study under analysis, with
a summary of the information concerning the same quantity available from the data collected in the study (known as the likelihood), to yield
an updated or posterior distribution of the quantity of interest.

These methods thus directly address the question of how new evidence should change what we currently believe. They extend naturally into making predictions, synthesising evidence from multiple sources, and designing studies: in addition, if we are willing to quantify the value of different consequences as a 'loss function', Bayesian methods extend into a full decision-theoretic approach to study design, monitoring and eventual policy decision-making. Nonetheless, Bayesian methods are a controversial topic in that they may involve the explicit use of subjective judgements in what is conventionally supposed to be a rigorous scientific exercise.

Objectives

This report is intended to provide:

a brief review of the essential ideas of Bayesian analysis
a full structured review of applications of Bayesian methods to randomised controlled trials, observational studies, and the synthesis of evidence, in a form which should be reasonably straightforward to update
a critical commentary on similarities and differences between Bayesian and conventional approaches
criteria for assessing the reporting of a Bayesian analysis
a comprehensive list of published 'three-star' examples, in which a proper prior distribution has been used for the quantity of primary interest
tutorial case studies of a variety of types
recommendations on how Bayesian methods and approaches may be assimilated into health technology assessments in a variety of contexts and by a variety of participants in the research process.

Methods

The BIDS ISI database was searched using the terms 'Bayes' or 'Bayesian'. This yielded almost 4000 papers published in the period 1990-98. All resultant abstracts were reviewed for relevance to health technology assessment; about 250 were so identified, and used as the basis for forward and backward searches. In addition EMBASE and MEDLINE databases were searched, along with websites of prominent authors, and available personal collections of references, finally yielding nearly 500 relevant references. A comprehensive review of all references describing use of 'proper' Bayesian methods in health technology assessment (those which update an informative prior distribution through the use of Bayes's theorem) has been attempted, and around 30 such papers are reported in structured form. There has been very limited use of proper Bayesian methods in practice, and relevant studies appear to be relatively easily identified.

Results

Bayesian methods in the health technology assessment context

Different contexts may demand different statistical approaches. Prior opinions are most valuable when the assessment forms part of a series of similar studies. A decision-theoretic approach may be appropriate where the consequences of a study are reasonably predictable.
The prior distribution is important and not unique, and so a range of options should be examined in a sensitivity analysis. Bayesian methods are best seen as a transformation from initial to final opinion, rather than providing a single 'correct' inference.
The use of a prior is based on judgement, and hence a degree of subjectivity cannot be avoided. However, subjective priors tend to show predictable biasses, and archetypal priors may be useful for identifying a reasonable range of prior opinion. For a prior to be taken seriously, its evidential basis must be explicitly given.
The Bayesian approach provides a framework for considering the ethics of randomisation.
Monitoring trials with sceptical and other priors may provide a unified approach to assessing whether the results of a trial should be convincing to a wide range of reasonable opinion, and could provide a formal tool for data-monitoring committees.
In contrast to earlier phases of development, it is generally unrealistic to formulate a Phase III trial as a decision problem, except in circumstances where future treatments can be accurately predicted.
Observational data will generally require more complex analysis: the explicit modelling of potential biasses may be widely applicable but needs some evidence-base in order to be convincing.
A unified Bayesian approach is applicable to a wide range of problems concerned with evidence synthesis, for example in pooling studies of differing designs in the assessment medical devices.
Priors for the degree of 'similarity' between alternative designs can be empirically informed by studies comparing the results of randomised controlled trials and observational data.
Increased attention to pharmaco-economics should lead to further investigation of decision-theoretic models for research planning, although this will not be straightforward.
Regulatory agencies are acknowledging Bayesian methods and have not ruled out their use, and the regulation of medical devices is leading the way in establishing the role of evidence synthesis.
'Comprehensive decision modelling' is likely to become increasingly important in policy making.
The BayesWatch criteria described in this report may provide a basis for structured reporting of Bayesian analysis.
Summaries of fully fledged ('three-star') applications of Bayesian methods in health technology assessment contain few prospective analyses but provide useful guidance.
Four case studies show:
1. Bayesian analyses using a sceptical prior can be useful to the data-monitoring committee of a cancer clinical trial.
2. Bayesian methods can be used to temper overoptimistic conclusions based on meta-analysis of small trials.
3. Modern graphical software can easily handle complex assessments previously analysed using the 'confidence profile' method.
4. Bayesian methods provide a flexible tool for performance estimation and ranking of institutions.

Recommendations and implications for future research and development

Bayesian methods could be of great value within health technology assessment, but for a realistic appraisal of the methodology, it is necessary to distinguish the roles and requirements for five main participant groups in health technology assessment: methodological researchers, sponsors, investigators, reviewers and consumers. Two common themes for all participants can immediately be identified. First, the need for an extended set of case studies showing practical aspects of the Bayesian approach, in particular for prediction and handling multiple substudies, in which mathematical details are minimised but details of implementation are provided. Second, the development of standards for the performance and reporting of Bayesian analyses, possibly derived from the BayesWatch checklist.

Some specific potential areas of research and development include:

Design. Realistic development of payback models and consideration of 'open' studies.
Priors. Investigation of evidence-based prior distributions appropriate to the participant group, as well as reasonable default priors in non-standard situations.
Modelling. Efficient use of all available evidence by appropriate joint modelling of historical controls, related studies, and so on.
Reporting. Development of criteria along the lines of the BayesWatch checklist, so that future users can reproduce analyses.
Decision-making. Increased integration with a health-economic and policy perspective, together with flexible tools for implementation.

Publication

Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. Bayesian methods in health technology assessment: a review. Health Technol Assess 2000;4(34).

NHS R&D HTA Programme

The NHS R&D Health Technology Assessment (HTA) Programme was set up in 1993 to ensure that high-quality research information on the costs, effectiveness and broader impact of health technologies is produced in the most efficient way for those who use, manage and provide care in the NHS.

Initially, six HTA panels (pharmaceuticals, acute sector, primary and community care, diagnostics and imaging, population screening, methodology) helped to set the research priorities for the HTA Programme. However, during the past few years there have been a number of changes in and around NHS R&D, such as the establishment of the National Institute for Clinical Excellence (NICE) and the creation of three new research programmes: Service Delivery and Organisation (SDO); New and Emerging Applications of Technology (NEAT); and the Methodology Programme.

Although the National Coordinating Centre for Health Technology Assessment (NCCHTA) commissions research on behalf of the Methodology Programme, it is the Methodology Group that now considers and advises the Methodology Programme Director on the best research projects to pursue.

The research reported in this monograph was funded as project number 93/50/05.

The views expressed in this publication are those of the authors and not necessarily those of the Methodology Programme, HTA Programme or the Department of Health. The editors wish to emphasize that funding and publication of this research by the NHS should not be taken as implicit support for any recommendations made by the authors.

Criteria for inclusion in the HTA monograph series
Reports are published in the HTA monograph series if (1) they have resulted from work commissioned for the HTA Programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees and editors.

Reviews in Health Technology Assessment are termed 'systematic' when the account of the search, appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the replication of the review by others.

Methodology Programme Director: Professor Richard Lilford
HTA Programme Director: Professor Kent Woods
Series Editors: Professor Andrew Stevens, Dr Ken Stein and Professor John Gabbay
Monograph Editorial Manager: Melanie Corris

The editors and publisher have tried to ensure the accuracy of this report but do not accept liability for damages or losses arising from material published in this report. They would like to thank the referees for their constructive comments on the draft document.

HTA Home Page | Details page for this publication | Publications listing | Publications search