Health Technology Assessment 1998 Vol. 2: No. 16. (Executive
monograph in Adobe Acrobat format (1036 kbytes)
4-page summary in Adobe Acrobat format (suitable for printing)
Qualitative research methods in health technology assessment: a review of the
School of Sociology and Social Policy, University of Nottingham, Nottingham, UK
Qualitative research involves the collection, analysis and interpretation of data that
are not easily reduced to numbers. These data relate to the social world and the concepts
and behaviours of people within it. Qualitative research can be found in all social
sciences and in the applied fields that derive from them, for example, research in health
services, nursing and pharmacy. These research methods are not a recent invention but
their application in health technology assessment (HTA) or health services research may be
novel. In order for commissioners and researchers to utilise these methods and gain
valuable knowledge from the results obtained, it is important that they understand the
principles of qualitative methods and the way that they may be used to set benchmark
The objective of this review was to examine the nature and status of qualitative
methods in relation to their potential uses in HTA.
The search tools often used for systematic reviews were not appropriate for this review as
it would be necessary to cover the equivalents of MEDLINE in a range of disciplines and
applied fields, many of which do not have databases of comparable coverage. In addition,
important methodological writing in the field of social science started long before
indexing for computer databases, and much of the most significant work has been published
in books rather than journals.
Having set the boundaries and organised the categories for this review, therefore, the
authors read as widely and as comprehensively as was feasible in the time available. The
authors have compared different researchers' approaches to the same issue and examined the
work of authors who offer different perspectives.
Idealists versus realists
Qualitative work is often identified with idealism while quantitative work is
identified with realism. However, most qualitative researchers accept that there is an
objective, material world, as do realists, but question our ability to know this directly.
In the social sciences, what people perceive or believe is the basis of their actions
rather than what an impartial observer might think was actually true.
Qualitative versus quantitative methods
The goal of all research in HTA should be to establish knowledge about which we can be
reasonably confident, and to provide findings that are relevant to policy makers and
practitioners. Therefore, decisions about whether qualitative or quantitative methods (or
a combination of both) are most appropriate to a particular research problem should be
made on the basis of which approach is likely to answer the question most effectively and
Qualitative methods are useful in the exploratory stages of a research project, where
they will often help to clarify or even set the research question, aid conceptualisation
and generate hypotheses for later research. Qualitative methods may also be used to
interpret, qualify or illuminate the findings of quantitative research and to test
Qualitative research is particularly useful to policy makers and planners by providing
descriptive information and understanding of the context in which policies will be
Sampling and generalising
In sampling decisions in qualitative research, pragmatic considerations should be
integrated with sampling in a systematic way just as in quantitative research;
opportunistic sampling should be avoided if possible. The emergent nature of qualitative
research means that sampling decisions need to be made throughout the study; such
decisions should again be systematic and principled.
Where the aim is to build or develop theory, subjects should be selected accordingly;
such theoretical sampling makes use of existing theory to make predictions, and then seeks
subjects who allow the researcher to test the robustness of such predictions under
Methods of qualitative research
Participant observation can be used to study the impact of technologies upon the routine
functioning of the setting in which they are to be implemented. Participant observation
may also be used to review health technologies currently in practice, and has the
potential for uncovering the process through which professional inputs are transformed
into patient/ client outcomes thereby identifying opportunities for modifying current
practice to improve outcomes.
Qualitative interview techniques are used, particularly in exploratory research, to study
the range and complexity of ideas and definitions employed by individuals and groups
involved in the implementation of health technologies. Both qualitative and quantitative
interviewing share the same fundamental problem, however, in that they rely upon
interviewees' reports and such reports are necessarily constrained by the context in which
they are collected.
The analysis of written records has an important contribution to make to our understanding
of the processes and consequences associated with new technologies. In addition, documents
such as health diaries may provide important data on the implementation of health
The techniques of conversation analysis can provide detailed data on the impact of new
technologies upon healthcare settings, the organisation of professional work and the
interactions between health professionals and patients.
The same ethical principles apply to qualitative and quantitative research in HTA. The
mechanical application of ethical codes developed in the context of biomedicine may be
unduly constraining in qualitative research and may distract from those ethical risks
which are specific to qualitative research. Covert research will rarely, if ever, be
justified in HTA. Such research is likely to be a betrayal of trust and a gross invasion
Assessment of qualitative research
The same assessment criteria of validity and relevance are appropriate for both
qualitative and quantitative research in HTA.
The relevance of HTA research is related to its potential generalisability to groups or
settings beyond those studied. Given that most qualitative research is based on a single
case or only a small number of subjects, the generalisability of qualitative research is
achieved by the generation of theoretical statements, which may guide policy makers but
remain to be tested through application in other contexts.
HTA commissioners should look for evidence that applicants intend to use systematic
methods for coding and handling their qualitative data and that methods proposed for
analysing such data are appropriate to the research objective.
Computerised analysis packages for qualitative data offer an efficient way of handling
qualitative data sets and may improve the rigour of the analysis by facilitating searches
for falsifying evidence. However, such programs should be used only as a means of
facilitating the analysis process rather than carrying out the analysis, which depends
upon the theoretical sensitivity of the analyst.
Judgements about the validity of research depend upon being able to form a judgement of
the research process. Researchers therefore need to provide a detailed record of their
methods. Given the non-standardised nature of qualitative research, such records are
likely to be more elaborate than in reports of quantitative research.
The trustworthiness of data analyses is enhanced where researchers can demonstrate that
they have considered alternative plausible explanations for their data. The validity of
research findings is enhanced where the researchers increase our understanding of all
members in a setting and do not present one-sided accounts. Confidence in the validity of
findings is increased where there is evidence of researcher sensitivity to the ways in
which the data have been shaped by the researcher's presence.
While the practices of respondent validation and triangulation may increase the
comprehensiveness of a study, neither can be treated as tests of the validity findings.
There are strengths and limitations to qualitative approaches as there are to
quantitative methods. However, where qualitative research is conducted properly and data
analysed thoroughly, this approach can provide valuable information on the implementation
and impact of health technologies on both health professionals and patients.
Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P. Qualitative
research methods in health technology assessment: a review of the literature. Health
Technol Assessment 1998; 2(16).
The overall aim of the NHS R&D Health Technology Assessment (HTA)
programme is to ensure that high quality research information on the costs, effectiveness
and broader impact of health technologies is produced in the most efficient way for those
who use, manage and work in the NHS. Research is undertaken in those areas where the
evidence will lead to the greatest benefits to patients, either through improved patient
outcomes or the most efficient use of NHS resources.
The Standing Group on Health Technology advises on national priorities
for health technology assessment. Six advisory panels assist the Standing Group in
identifying and prioritising projects. These priorities are then considered by the HTA
Commissioning Board supported by the National Coordinating Centre for HTA.
This report is one of a series covering acute care, diagnostics and
imaging, methodology, pharmaceuticals, population screening, and primary and community
care. It was identified as a priority by the Methodology Panel and funded as project
The views expressed in this publication are those of the authors and
not necessarily those of the Standing Group, the Commissioning Board, the Panel members or
the Department of Health. The editors wish to emphasize that funding and publication of
this research by the NHS should not be taken as implicit support for the recommendations
for policy contained herein. In particular, policy options in the area of screening will,
in England, be considered by the National Screening Committee. This Committee, chaired by
the Chief Medical Officer, will take into account the views expressed here, further
available evidence and other relevant considerations.
Reviews in Health Technology Assessment are termed 'systematic' when
the account of the search, appraisal and synthesis methods (to minimise biases and random
errors) would, in theory, permit the replication of the review by others.
NHS R&D HTA Programme
The editors have tried to ensure the accuracy of this report but cannot
accept responsibility for any errors or omissions. They would like to thank the referees
for their constructive comments on the draft document.
Andrew Stevens, Ruairidh Milne, Ken Stein
©1998 Crown Copyright