HTA Home Page | Details page for this publication | Publications listing | Publications search

Health Technology Assessment 2002; Vol. 6: No. 32

Executive summary

View/Download full monograph in Adobe Acrobat format (1012 kbytes)
View/Download this 4-page summary in Adobe Acrobat format (suitable for printing)

The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature

R Crow¹
H Gage^2*
S Hampson³
J Hart⁴
A Kimber⁵
L Storey¹
H Thomas⁶

¹ European Institute of Health and Medical Sciences, University of Surrey, Guildford, UK
² Department of Economics, University of Surrey, Guildford, UK
³ Department of Psychology, University of Surrey, Guildford, UK
⁴ School of Psychology, University of St Andrews, UK
⁵ School of Applied Statistics, University of Reading, UK
⁶ Department of Sociology, University of Surrey, Guildford, UK

^* Corresponding author

Background

Satisfaction and its measurement are important for public policy analysts, healthcare managers, practitioners and users. Despite problems with establishing a tangible definition of “satisfaction” and difficulties with its measurement, the concept continues to be widely used. In many instances when investigators claim to be measuring satisfaction, more general evaluations of healthcare services are being undertaken.

Satisfaction can be measured indirectly by asking users to rate the quality of services they have received, or report their experiences. Selection (or deselection) of providers is an objective behavioural indicator of satisfaction in healthcare systems where consumers’ choices are not constrained. Healthcare is a multi-dimensional service, but many means of measuring satisfaction do not show consumers’ relative preferences for different attributes, even though such information is important for cost-effective decision-making.

Objectives

The review sought to:

summarise the results of studies that investigated methodological issues
identify determinants of satisfaction with healthcare in different settings
explore gaps in existing knowledge so that they can be addressed by future research
consider the implications of the findings for the NHS.

Better information on the factors affecting satisfaction will assist healthcare providers and planners in the NHS to improve the quality of the service they deliver to users. Guidance on methods of collecting feedback from consumers will ensure that reliable information for the decision-making process may be collected.

Methods

Electronic searching involved seven major databases covering the years 1980–2000 and a range of terms. Foreign language articles were not excluded. Non-electronic search strategies involved outreach activities to a wide range of organisations, and personal contacts with leading academics in the field.

The review was conducted in two phases: an initial search resulted in the analysis of 128 articles, and a further 48 articles were added as a result of exploding reference lists and updating the electronic search.

Over 3000 abstracts were screened for relevance by three team members. Articles were excluded if the evidence they contained was not generalisable. In particular, evaluations that were specific to disease groups or service delivery locations were rejected. The articles retained were categorised as: background (n = 190, including reviews, and conceptual and policy articles); empirical (n = 223, providing primary research evidence for analysis in the review); and instrument related (n = 92).

Data were extracted from empirical articles by one reader and checked by a second. To assess the methodological quality of studies, both readers independently completed quality assessment forms based on agreed criteria. Articles deemed as poor by both assessors were subsequently excluded (n = 47).

Summary tables were prepared for all included studies, and data were synthesised using SPSS spreadsheets. Articles were subjected to a narrative review owing to the variety of approaches and outcome measures.

Results: evidence on methodological issues

The review identified 37 studies (36 data sets) that addressed methodological issues.

With respect to modes and response rates, the review showed that:

Interview methods (telephone and face-to-face) generate higher responses than mail surveys (up to 30%).

Differences between mail and telephone response rates can be significantly reduced by telephone follow-up of mail non-respondents, although this adds to costs.

Mail is cheaper than interview, except where telephone follow-up is used.

Impersonal and mail methods result in more criticism/less reported satisfaction because respondents’ anonymity is not compromised and there is no pressure for socially acceptable responding.

Mail methods give more variability in responses (people feeling strongly either way tend to respond), but there may be concealed proxies.

The evidence on the effect of survey timing on reported satisfaction is contradictory, and may depend on the nature of the illness and extent of recovery.

Low response rates introduce bias. Non-respondents are more likely to be members of visible minority groups, less well educated and uninsured (in the USA).

There are problems with obtaining usable responses from elderly, severely ill and cognitively impaired people, and those with language problems.

On-site surveys under-represent low users in ambulatory populations.

Qualitative approaches are more resource intensive but access in-depth information not captured by structured questionnaires.

With respect to survey design issues, the review showed that:

Items with a personal referent are associated with higher recorded satisfaction than similarly worded items with a general referent.
Questionnaire design issues, such as scaling and wording, affect responses; acquiescent responding and cross-cultural differences can introduce bias.
Research on design issues is patchy and ad hoc.

Results: evidence on determinants of satisfaction

The review identified 139 articles (127 data sets) that provided evidence about the determinants of satisfaction. For analytical purposes, two groups of factors affecting satisfaction were identified: those relating to the characteristics of respondents, and those relating to health service delivery factors (policy variables).

The review showed that:

Despite the potential importance of expectations in the measurement of satisfaction, only 20% of studies considered this factor, with varied results, such that many questions remain unanswered in this area.
Satisfaction is linked to prior satisfaction with healthcare, respondents’ predisposition, utilisation, and granting patients’ desires (e.g. for tests and medications).
Health status and health outcomes affect satisfaction. In general, sicker patients and those experiencing psychological distress record lower satisfaction, with the possible exception of some chronically ill groups.
Older respondents generally record higher satisfaction, but evidence about the effects of gender, ethnicity and socio-economic status is equivocal.
There is consistent evidence across settings that the most important health service factor affecting satisfaction is the patient–practitioner relationship, including information giving.
Choice of service provider is associated with higher satisfaction. In the USA, care provided under fee-for-service arrangements generates greater satisfaction than that delivered by prepaid schemes, and gatekeeping arrangements score relatively poorly on satisfaction.

Conclusions

Areas identified where further research may be warranted

A review of the effects of satisfaction on health behaviours and health outcomes is needed in order to establish the importance to health services, and to individuals, of fostering satisfaction.
With respect to methodological issues, research is needed on:
- the effect of timing of surveys on reported satisfaction
- the extent of bias introduced by interviewers
- cross-cultural issues and adaptations
- how consumer feedback can be incorporated into healthcare decision-making, including the development of measures of relative preference.
With respect to the role of expectations, research is needed to:
- classify different types of expectations and explore how consumers operationalise these in evaluations
- identify influences on expectations
- examine the relationship between sociodemographic factors and expectations.
There is a need to explore how different types of illnesses and health outcomes affect evaluations.
Research is needed to explore the effect of different incentive structures on physician behaviour and patient satisfaction.

Methodological recommendations

Researchers seeking to collect high-quality information about consumers’ views should pay particular attention to:

how different ways of conducting surveys affect response rates and consumers’ evaluations
sampling methods, particularly to include disadvantaged groups
promoting high response rates, to protect against low response bias
the effect of respondents’ expectations, prior experiences and desires
establishing the strength of relative preferences between attributes because this has advantages in a policy-making context, particularly with cost-effectiveness considerations in mind.

Resource considerations are likely to be an important influence on the choice of survey method. Investigators must determine, in the context of their own requirements, whether the extra benefits derived from more costly approaches are worth the extra expenditure.

Qualitative approaches provide in-depth information, in contrast to the reductionism implied by quantitative approaches. Open-ended questioning in structured questionnaires may be a compromise. If resources permit, a mix of approaches may be optimal.

Recommendations from the review of the determinants literature

If consumer satisfaction is a priority, there is a need systematically to address interpersonal issues in the training of all staff, and to ensure that the financial and regulatory arrangements encourage practitioners to foster supportive and interactive relationships with their patients.

There is evidence that age and health status can affect consumers’ ratings; these non-policy factors should be borne in mind when interpreting the results of satisfaction surveys.

Implications of the review for the NHS

Information gathering from NHS consumers about their satisfaction, or causes of dissatisfaction, is essential to the quality assurance process because limited choice means that preferences cannot be expressed by changing providers. The review addressed methodological issues to help managers and practitioners to collect reliable information from users about their views. The evidence on the determinants of satisfaction has been synthesised. User interests will be served only if their feedback affects decision makers. Complaints data are not comprehensive.

The National Plan for the NHS requires local managers and practitioners to conduct surveys of consumers’ views. The results of these surveys could be used in national level performance indicators. Evaluation of the costs and consequences of alternative means of collecting feedback would be beneficial, in particular the balance between local and national needs, and between qualitative and quantitative approaches. Instrument design is costly and complex, but some validated instruments are available.

Publication

Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, et al. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess 2002;6(32).

NHS R&D HTA Programme

The NHS R&D Health Technology Assessment (HTA) Programme was set up in 1993 to ensure that high-quality research information on the costs, effectiveness and broader impact of health technologies is produced in the most efficient way for those who use, manage and provide care in the NHS.

Initially, six HTA panels (pharmaceuticals, acute sector, primary and community care, diagnostics and imaging, population screening, methodology) helped to set the research priorities for the HTA Programme. However, during the past few years there have been a number of changes in and around NHS R&D, such as the establishment of the National Institute for Clinical Excellence (NICE) and the creation of three new research programmes: Service Delivery and Organisation (SDO); New and Emerging Applications of Technology (NEAT); and the Methodology Programme.

Although the National Coordinating Centre for Health Technology Assessment (NCCHTA) commissions research on behalf of the Methodology Programme, it is the Methodology Group that now considers and advises the Methodology Programme Director on the best research projects to pursue.

The research reported in this monograph was funded as project number 96/27/02.

The views expressed in this publication are those of the authors and not necessarily those of the Methodology Programme, HTA Programme or the Department of Health. The editors wish to emphasize that funding and publication of this research by the NHS should not be taken as implicit support for any recommendations made by the authors.

Criteria for inclusion in the HTA monograph series
Reports are published in the HTA monograph series if (1) they have resulted from work commissioned for the HTA Programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees and editors.

Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search, appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the replication of the review by others.

Methodology Programme Director: Professor Richard Lilford
HTA Programme Director: Professor Kent Woods
Series Editors: Professor Andrew Stevens, Dr Ken Stein, Professor John Gabbay, Dr Ruairidh Milne and Dr Chris Hyde
Managing Editors: Sally Bailey and Sarah Llewellyn Lloyd

The editors and publisher have tried to ensure the accuracy of this report but do not accept liability for damages or losses arising from material published in this report. They would like to thank the referees for their constructive comments on the draft document.

HTA Home Page | Details page for this publication | Publications listing | Publications search