R Fitzpatrick,^1* J Chambers,¹ T Burns,² H Doll,¹ S Fazel,² C Jenkinson,¹ A Kaur,¹ M Knapp,³ L Sutton² and J Yiend⁴

¹Department of Public Health, University of Oxford, Oxford, UK
²Department of Psychiatry, University of Oxford, Oxford, UK
³Personal Social Services Research Unit, London School of Economics, London, UK
⁴Division of Psychological Medicine and Psychiatry, Institute of Psychiatry, King’s College London, London, UK

*Corresponding author

Introduction

This study examines outcome measurement in forensic mental health research. Forensic mental health services cover many domains such as prisons, community corrections and secure forensic hospitals. Within this complex system each service uses outcome measures for its own specific objectives, with little standardisation between organisations. Outcome measurement is also difficult to standardise as the client population often suffers from multiple problems including mental health disorders and substance abuse, leading to multiple targets for intervention. Research in forensic mental health also suffers problems such as a moving population, priorities to maintain security and duty of care to clients, preventing extensive application of studies based on randomised controlled trials. Overall, there is very little methodological discussion about outcome measurement in forensic mental health research.

Objectives

This research project was exploratory in nature, to describe and assess outcome measures in forensic mental health research. A two-stage study was conducted. Stage one consisted of a structured review of outcome measures used in forensic mental health research. Stage two consisted of a consensus panel that considered the essential domains of outcome measurement in forensic mental health and then assessed the properties of the most frequently used outcome measures against key questions. The panel included experts from within forensic mental health research and services.

Methods

Structured review

A search of eight electronic databases, including CINAHL, EMBASE and MEDLINE, was conducted for the period 1990–2006. Search and medical subject heading terms focused upon two factors: the use of a forensic participant sample and the experimental designs likely to be used for outcome measurement.

Studies eligible for inclusion in the review fulfilled several criteria:

1. Participants were defined as offenders or residents of a forensic mental health institution.
2. The study required examination of an intervention with the use of outcome measurement after the intervention.
3. Study design was required to be either a randomised controlled trial or a quasi-experimental (comparing intervention and control) design with a minimum follow-up of 6 months.
4. A mental health element needed to be present in the participant population, the intervention or an outcome measurement.

Data extraction included general information about the identity of the reference, specific information regarding the study and information pertaining to the outcome measures used.

Data about mental health research outcome measures were extracted from the included references, and were entered into EXCEL. The outcome measures that occurred most frequently were also calculated.

Consensus group

The consensus exercise was implemented in two stages. At the first stage, participants were asked to complete ratings about the importance of various potential areas of outcome measurement (‘domains’) in a written consultation. At the second stage, they were asked to attend a consensus meeting to review and agree results relating to the domains, to consider and rate specific outcome instruments identified as commonly used from the structured review and to discuss strengths, weaknesses and future priorities for outcome measurement in forensic mental health research. Participants comprised three representatives from psychology, three from psychiatry and one from each of criminology, probation, prison health and nursing.

Results

The final sample of eligible studies for inclusion in the review consisted of 308 separate studies obtained from 302 references.

The consensus group agreed on 11 domains of forensic mental health outcome measurement, all of which were considered important. In the literature review, 1038 distinct variables were identified that were used to assess outcomes in the sample of evaluative studies. Nine different outcome measure instruments were used in more than four different studies. The most frequently used outcome measure was used in 15 studies. A further review of research concerning the psychometric properties of these instruments was carried out. It revealed little evidence specifically to validate their use with forensic populations. The measures that were rated most favourably by the consensus panel were the Beck Depression Inventory, the Brief Psychiatric Rating Scale and the Symptom Checklist-90-Revised. According to the consensus group, many domains beyond recidivism and mental health were important but under-represented in the review of outcomes. Current instruments that may show future promise in outcome measurement included risk assessment tools.

The outcome measure of repeat offending behaviour was by far the most frequently used, occurring in 72% of the studies included in the review. Its measurement varied with position in the criminal justice system, offence specification and method of measurement. The consensus group believed that recidivism is only an indication of the amount of antisocial acts that are committed.

Conclusions and recommendations

A wide range of domains are relevant to assessing outcomes of interventions in forensic mental health services. Evaluations need to take account of public safety, but also clinical, rehabilitation and humanitarian outcomes. To date, research has focused extensively on the first domain, evaluating outcomes in terms of recidivism.

Recidivism is a very high priority; the public expects interventions that will reduce future criminal behaviour. Greater attention needs to be given to validity of measurement, given the enormous variety of approaches to measurement. More research is needed on methods to take account of the heterogeneity of seriousness of forms of recidivism in outcome measurement. Validity of self-report instruments regarding recidivism also needs examination by further research.

Mental health is clearly also an important dimension of outcome. Instruments have been used in forensic mental health research that have been well validated in the context of general mental health research.

The review provides clear support for the view that domains such as quality of life, social function and psychosocial adjustment have not been extensively employed in forensic mental health research, but are relevant and important issues. The role of such instruments needs more consideration. Research is needed in these domains to complement the evidence base of outcomes in terms of public safety and mental health.

The wide array and diversity of measures used in forensic mental health research suggests that there is still substantial scope for standardisation, by further use of consensus-type processes to identify domains and specific measures that are relevant to and familiar in practice and can be more widely used in evaluative research.

The role of instruments assessing dynamic aspects of the risk of violence offer a particular opportunity. They are becoming more widely known in practice and could be more widely used in evaluative research in this field.

© 2010 Crown Copyright Top ^

Details page for this publication