Health Technology Assessment 2005; Vol 9: number 38
Executive Summary
View/Download full monograph in Adobe Acrobat format
(950 kbytes)
View/Download summary in Adobe Acrobat format (suitable for printing)
The causes and effects of socio-demographic exclusions from clinical trials
C Bartlett,1 L Doyal,2 S Ebrahim,1 P Davey,3 M Bachmann,4 M Egger5 and P Dieppe1*
1 Medical Research Council Health
Services Research Collaboration, Department of Social Medicine, University of
Bristol, UK
2 School for Policy Studies, University of Bristol, UK
3 MEMO, Department of Clinical Pharmacology, University of
Dundee, Ninewells Hospital, Dundee, UK
4 School of Medicine, Health Policy and Practice, University of
East Anglia, Norwich, UK
5 Department of Social and Preventative Medicine, University of
Berne, Switzerland
* Corresponding author
Background
The exclusion from trials of people likely to be in need of or to benefit from an intervention could compromise the trials’ generalisability. We investigated the exclusion of women, older people and minority ethnic groups, focusing on two drug exemplars, statins and non-steroidal anti-inflammatory drugs (NSAIDs).
Objectives
- Scope the social, legal and ethical contexts of trial exclusion, comparing the UK with the USA.
- Document disparities between people included in trials, those using the drugs and those in need of the treatment.
- Project the effects of exclusion on the generalisability of trials, referring to effectiveness (statins) and adverse effects (NSAIDs).
- Develop a theoretical model for the causes and effects of exclusions.
Methods
Scoping
We reviewed literature on the exclusion of women, older people and ethnic minorities in healthcare research and held three workshops with stakeholders.
Trials
We analysed 27 randomised controlled trials (RCTs) of statins use for secondary prevention of coronary heart disease (CHD), lasting at least 6 months (up to August 2001). We analysed a stratified sample of 25 NSAIDs trials for pain in osteoarthritis (OA) (up to 1998, prior to the introduction of coxibs).
Cohorts
Using a Scottish cohort with record-linkage [Medicines Monitoring Unit (Dundee) (MEMO)], we profiled 3188 people needing secondary prevention for CHD (1993–1996), ascertaining the independent effects of statins, and 131,410 people dispensed NSAIDs (1989–1996), examining adverse effects.
Use and need
To profile the need for secondary prevention of CHD in England we accessed routine data sources including Hospital Episode Statistics (HES). To estimate usage we consulted published surveys. For potential need and usage of NSAIDs in OA we accessed the Somerset and Avon Survey of Health (SASH) 1996–97 and published data.
Disparities
For both drugs, we compared the socio-demographic profiles of trial samples, the population in potential need and those on treatment.
Epidemiological/statistical assumptions
We produced an evidence synthesis to clarify the effects of statins on women and older people. We modelled the relationship of absolute effectiveness outcomes (e.g. numbers needed to treat) with underlying risk levels of disease events, examining the likely effects of trial exclusions.
Results
Scoping
In the USA, the discourse has expanded from protecting the vulnerable to include justice and the equitable access of different groups to trials. Appropriate representation of women and ethnic minorities in publicly funded trials is required by legislation. Guidelines recommend appropriate inclusion by age. In the UK, the debate is more limited, and equity and inclusivity in research are not formally promoted.
Trials
Statins
The average age of trial participants was 58.5 years; only 16.3% were women. Statins reduced cardiovascular disease (CVD) incidence by about 25% in both men and women. Older people up to 75 years of age also benefited. Meta-analysis and two landmark trials, containing large proportions of women and older people (published after 2001), confirmed these results.
NSAIDs
The average age of trial participants was 61.9 years and women were well represented (68.5%). Gastrointestinal (GI) adverse events were commonly reported, but renal side-effects were not. Outcomes were seldom reported according to socio-demographic group.
For both drugs, USA trials were more inclusive than UK/European trials. Ethnicity was not well reported for either drug.
Cohorts
Statins
Some 23% of the cohort were treated with statins. Statins users were younger than non-statins users (but no more likely to be male) and had superior outcomes.
NSAIDs
High current exposure to NSAIDs elevated the risk of GI side-effects by about 50% versus no current exposure and renal impairment risk by nearly 140%. Side-effect risk increased with age; being female diminished risk.
Use and need
Statins
Approximately 537,000 incident cases of CVD would qualify for statins use in England each year. Women constitute 45% of this population with need, two-thirds of whom are aged 65 years or over. Need varies by ethnic group. No sex bias in prescribing was detected, but use was commoner in younger people.
NSAIDs
6.3% of adults aged 35+ years reported hip and/or knee pain associated with OA; 3.9% of adults used prescribed analgesics for this and they were more likely to be women and to be >65 years old.
Disparities
Statins
Women formed almost half of the ‘with need’ and ‘on treatment’ populations, but were markedly under-represented in trials. Those aged 65+ years formed nearly two-thirds of the ‘with need’ population, but only one-fifth of trial samples, and were less likely to be treated than younger subjects.
NSAIDs
Women formed similar proportions (two-thirds) of trial samples, and of the ‘with need’ and ‘on treatment’ populations. People aged 65+ years formed about three-fifths of the ‘on treatment’ population, but were under-represented in trials. Association of side-effects with socio-demographic factors was revealed in cohort data but not in trials.
Epidemiological/statistical assumptions
Meta-analysis might overcome problems of low inclusion for the assessment of relative effectiveness, but the assessment of side-effects in different groups would require massive trials. Measures of absolute effectiveness are vital for the analyses of benefit and harm and cost-effectiveness. Such measurements, involving underlying risk levels, will be severely biased if different population groups are not adequately represented.
Main conclusions
The issue of exclusion from trials of women, older people and ethnic minorities has been relatively neglected in the UK research community, and there is confusion about diversity issues. Under-representation occurs, but in drug trials at least this may not always affect the external validity of relative effect estimates. However, measures of absolute effectiveness, absolute harm and cost-effectiveness are associated with underlying risk levels in different socio-demographic groups. Under-representation will therefore bias absolute effect estimates. The complexity of the issues made development of a theoretical model impossible.
Recommendations for future research
The following areas are suggested for future research:
- Multi-disciplinary assessment of realistic options for trialists to address the issue of exclusions.
- Clarification of the use of ethnic categories in health research and of the implications of the different dimensions of ageing and sex/gender.
- Identification of barriers and facilitators to the involvement of different population groups in research.
- Further investigation of the susceptibility of older men to NSAID adverse events.
- Development of a ‘register of registries and databases’ and exploration of how linked health information systems in the UK could be improved.
Publication
Bartlett C, Doyal L, Ebrahim S, Davey P, Bachmann M, Egger M, et al. The causes and effects of socio-demographic exclusions from clinical trials. Health Technol Assess2005;9(38).
NHS R&D HTA Programme
The research findings from the NHS R&D Health Technology Assessment (HTA) Programme directly influence key decision-making bodies such as the National Institute for Health and Clinical Excellence (NICE) and the National Screening Committee (NSC) who rely on HTA outputs to help raise standards of care. HTA findings also help to improve the quality of the service in the NHS indirectly in that they form a key component of the ‘National Knowledge Service’ that is being developed to improve the evidence of clinical practice throughout the NHS.
The HTA Programme was set up in 1993. Its role is to ensure that high-quality research information on the costs, effectiveness and broader impact of health technologies is produced in the most efficient way for those who use, manage and provide care in the NHS. ‘Health technologies’ are broadly defined to include all interventions used to promote health, prevent and treat disease, and improve rehabilitation and long-term care, rather than settings of care.
The HTA Programme commissions research only on topics where it has identified key gaps in the evidence needed by the NHS. Suggestions for topics are actively sought from people working in the NHS, the public, service-users groups and professional bodies such as Royal Colleges and NHS Trusts.
Research suggestions are carefully considered by panels of independent experts (including service users) whose advice results in a ranked list of recommended research priorities. The HTA Programme then commissions the research team best suited to undertake the work, in the manner most appropriate to find the relevant answers. Some projects may take only months, others need several years to answer the research questions adequately. They may involve synthesising existing evidence or conducting a trial to produce new evidence where none currently exists.
Additionally, through its Technology Assessment Report (TAR) call-off contract, the HTA Programme is able to commission bespoke reports, principally for NICE, but also for other policy customers, such as a National Clinical Director. TARs bring together evidence on key aspects of the use of specific technologies and usually have to be completed within a short time period.
Criteria for inclusion in the HTA monograph series
Reports are published in the HTA monograph series if (1) they have resulted from work commissioned for the HTA Programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees and editors.
Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search, appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the replication of the review by others.
The research reported in this monograph was commissioned by the HTA Programme as project number 98/24/02. The contractual start date was in June 2001. The draft report began editorial review in August 2003 and was accepted for publication in March 2005. As the funder, by devising a commissioning brief, the HTA Programme specified the research question and study design. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.
The views expressed in this publication are those of the authors and not necessarily those of the HTA Programme or the Department of Health.
Editor-in-Chief: Professor Tom Walley
Series Editors: Dr Peter Davidson, Dr Chris Hyde,
Dr Ruairidh Milne, Dr Rob Riemsma and Dr Ken Stein
Managing Editors: Sally Bailey and Sarah Llewellyn Lloyd
© 2005 Crown Copyright Top ^