R Baker,^1,2 I Bateman,³ C Donaldson,^1,2* M Jones-Lee,⁴ E Lancsar,^1,4 G Loomes,⁵ H Mason,¹ M Odejar,⁶ JL Pinto Prades,^7,8 A Robinson,⁹ M Ryan,⁶ P Shackley,¹⁰ R Smith,¹¹ R Sugden⁵ and J Wildman⁴ (the SVQ Research Team)

¹Institute of Health and Society, Newcastle University, Newcastle upon Tyne, UK
²Yunus Centre, Glasgow Caledonian University, Glasgow, UK
³School of Environmental Sciences, University of East Anglia, Norwich, UK
⁴Newcastle University Business School, Newcastle upon Tyne, UK
⁵School of Economics, University of East Anglia, Norwich, UK
⁶Health Economics Research Unit, University of Aberdeen, Aberdeen, UK
⁷Department of Economics, University Pablo de Olavide, Sevilla, Spain
⁸Fundación Centro de Estudios Andaluces, Sevilla, Spain
⁹School of Medicine, Health Policy and Practice, University of East Anglia, Norwich, UK
¹⁰School of Health and Related Research, University of Sheffield, Sheffield, UK
¹¹Department of Public Health and Policy, London School of Hygiene & Tropical Medicine, London, UK

*Corresponding author

The National Institute for Clinical Excellence (NICE) was created in 1999 and renamed the National Institute for Health and Clinical Excellence (but still NICE) in 2005. One of NICE’s major roles is to advise the rest of the National Health Service (NHS) in England about the adoption and retention of health interventions. In assessing health technologies, the NICE process involves the use of economic evaluations. These evaluations reflect the dominant paradigm in health economics, with new technologies assessed, usually relative to current practice, in terms of additional cost per quality-adjusted life-year (QALY) gained. Since the inception of NICE, therefore, and even in the wider health economics literature prior to 1999, this has raised the questions of whether QALYs gained by different beneficiaries of health care should be weighted equally and whether it is feasible to estimate a monetary value of a QALY.

The Social Value of a QALY (SVQ) Team based at Newcastle University and the Universities of East Anglia and Aberdeen were contracted from October 2004 to September 2007 to undertake two studies, each based on a survey of the population in England.

The first study was the ‘relativities study’, which aimed to identify characteristics of beneficiaries of health care over which relative weights are to be derived, and to estimate the relative weights to be attached to health gains according to the characteristics of the recipients of these gains.

The second study was the ‘valuation study’, which aimed to assess the feasibility of estimating a willingness-to-pay (WTP)-based value of a QALY.

Relativities study

To identify characteristics of beneficiaries, we took a predominantly qualitative approach, supplementing conventional qualitative methods with other techniques. This exploratory part of the project was an iterative process involving several phases and a range of methods including: open-ended discussion in focus groups; simple ranking procedures; experimentation with sample questions; and a more complex ranking task, called Q methodology, involving card sorting. Qualitative findings were interpreted alongside the results of the other methods used.

This exploratory work, which lasted for over a year, taken together with considerations of policy relevance in consultation with representatives of NICE, resulted in the selection of the following attributes: age (both at onset of illness and at death) and severity of illness (with and without treatment). These were presented to respondents along with additional information representing gains in QALYs. They were put forward in the form of two types of question: discrete choice and matching.

Methods for presenting information about age, severity and health gain were devised in focus groups and tested in cognitive interviews. The most successful technique was achieved through the use of diagrams, which were first explained using an innovative, animated Microsoft POWERPOINT presentation. Respondents were then presented with pairs of scenarios in a series of choices, examples of which are contained in the main body of this report.

The discrete choice and matching questions were part of a longer questionnaire (including attitudinal and sociodemographic questions), which was administered face to face using a computer-assisted personal interview. The survey was administered to a nationally-representative sample (n = 587) of the population in England by the National Centre for Social Research (NatCen) from February to April 2007.

Two innovative approaches were then applied to the derivation of weights:

• A discrete choice approach, whereby, having presented respondents with a series of pairwise choices, each comprising combinations of age, severity levels and components of the QALY, a function for the relative importance of these attributes was estimated, this function being used as the basis of two approaches to deriving weights for age and severity.
• The ‘QALY grid’ approach, based on ‘matching’ (or ‘person trade-off’ questions), presenting respondents with a series of iterative pairwise choices over scenarios described in terms of age, severity levels and QALY components, where either age or severity was allowed to vary within a choice set. The scenarios in each set were initially described as having equal numbers of beneficiaries. Once an initial choice was made, the number of beneficiaries in the preferred option was reduced until the respondent was indifferent between alternatives presented. A set of questions was devised to allow derivation of relative weights for 20 different areas of the ‘grid’, covering five ranges of quality of life (from death to full health) and four age groups (between birth and 80 years of age).

The results of the discrete choice study show that age and severity variables did not have a strong impact on respondents’ choices over and above the health (QALY) gains presented. In contrast, the QALY grid analysis of the matching data showed clear evidence of both age and severity impacts, although measures of these varied with the aggregation method: the more conservative method of aggregation showed gains to some groups being weighted up to 2.75 times more highly than gains to others, whereas an alternative method gave ratios that could differ by up to 4 : 1. Still, the patterns in the matching data are robust to the choice of method: there is a general tendency to give greater weight to younger people and those in poorer health, although somewhat less weight is given to the very youngest and those in the very poorest health with limited prospects of improvement.

Valuation study

The valuation study was a feasibility study conducted on a smaller convenience sample (n = 409). Respondents were asked about their WTP to avoid/prevent different durations of head pain or stomach illness, and to value these states on a 0–1 scale (death = 0; full health = 1) using a set of standard gamble (SG) questions.

Results from the WTP and SG questions were combined in different ways to arrive at values of a QALY. Depending on the aggregation procedure, these vary from values which are in the vicinity of the current NICE threshold to extremely high values.

Conclusions

Implications for practice are limited because of the methodological nature of the research. The two main recommendations are as follows:

• On relativities, it could be said that it would be premature to propose any particular set of QALY weights at this point in time: before that point is reached, there is scope for both further reconciliation and replication. However, it might equally be argued that there is no scope for reconciliation and that we need to choose between the results in light of the caveats of the matching and discrete choice methods used.
• On valuation, it was never the intention to conduct a representative survey using a definitive method. The main recommendation, therefore, is that any future national sample survey should be preceded by further extensive qualitative research and cognitive testing to resolve the main questions identified in the present study.

The research recommendations arising from the study are as follows:

• The findings from the relativities study indicate that more work is required in the short term to reconcile the results obtained, although fundamental differences between the methods and results reported may challenge such reconciliation.
• In the longer term, and still with respect to relativities, further methodological research should attempt to account for some of the deficiencies of the methods (especially the particular discrete choice approach used in SVQ).
• Building on the results of the innovative methods that have been devised in this study to derive relative weights, further replication of these results is required to address this important policy issue.
• With respect to valuation, shorter-term work is required around the issues of aggregation, combining WTP and SG values and the appropriateness of different measures of central tendency.

In the longer term, more qualitative and cognitive research is required around two issues in particular: first, the problem of identifying health states to present to respondents which are ‘minor enough’ for people to be able to express their willingness to pay, but not so minor that respondents will accept only minuscule risks of death when responding to SG type questions; and second, but related to the first, the extent to which ‘noise’ and ‘error’ in people’s responses might generate extreme and unreliable figures.

Publication

Baker R, Bateman I, Donaldson C, Jones-Lee M, Lancsar E, Loomes G, et al. Weighting and valuing quality-adjusted life-years using stated preference methods: preliminary results from the Social Value of a QALY Project. Health Technol Assess 2010;14(27).

NIHR Health Technology Assessment programme

The Health Technology Assessment (HTA) programme, part of the National Institute for Health Research (NIHR), was set up in 1993. It produces high-quality research information on the effectiveness, costs and broader impact of health technologies for those who use, manage and provide care in the NHS. ‘Health technologies’ are broadly defined as all interventions used to promote health, prevent and treat disease, and improve rehabilitation and long-term care.

The research findings from the HTA programme directly influence decision-making bodies such as the National Institute for Health and Clinical Excellence (NICE) and the National Screening Committee (NSC). HTA findings also help to improve the quality of clinical practice in the NHS indirectly in that they form a key component of the ‘National Knowledge Service’.

The HTA programme is needs led in that it fills gaps in the evidence needed by the NHS. There are three routes to the start of projects.

First is the commissioned route. Suggestions for research are actively sought from people working in the NHS, from the public and consumer groups and from professional bodies such as royal colleges and NHS trusts. These suggestions are carefully prioritised by panels of independent experts (including NHS service users). The HTA programme then commissions the research by competitive tender.

Second, the HTA programme provides grants for clinical trials for researchers who identify research questions. These are assessed for importance to patients and the NHS, and scientific rigour.

Third, through its Technology Assessment Report (TAR) call-off contract, the HTA programme commissions bespoke reports, principally for NICE, but also for other policy-makers. TARs bring together evidence on the value of specific technologies.

Some HTA research projects, including TARs, may take only months, others need several years. They can cost from as little as £40,000 to over £1 million, and may involve synthesising existing evidence, undertaking a trial, or other research collecting new data to answer a research problem.

The final reports from HTA projects are peer reviewed by a number of independent expert referees before publication in the widely read journal series Health Technology Assessment.

Criteria for inclusion in the HTA journal series

Reports are published in the HTA journal series if (1) they have resulted from work for the HTA programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees and editors.

Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search, appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the replication of the review by others.

The research reported in this issue of the journal was commissioned by the National Coordinating Centre for Research Methodology (NCCRM), and was formally transferred to the HTA programme in April 2007 under the newly established NIHR Methodology Panel. The HTA programme project number is 06/91/05. The contractual start date was in October 2004. The draft report began editorial review in March 2009 and was accepted for publication in June 2009. The commissioning brief was devised by the NCCRM who specified the research question and study design. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.

Editor-in-Chief: Professor Tom Walley CBE
Series Editors: Dr Martin Ashton-Key, Dr Aileen Clarke, Professor Chris Hyde, Dr Tom Marshall, Dr John Powell, Dr Rob Riemsma and Professor Ken Stein