Police Drug Diversion (PDD): A Realist Impact, Process and Economic Evaluation

The project is led by The University of Kent with partners from another 4 universities, the College of Policing, and a charity (User voice). The police forces involved in the project are Durham / West Midlands and Thames Valley Police (OU partner)

Aims

Our realist evaluation aims to inform policy and practice on whether and how to roll out PDD schemes across the country. Our approach is as follows:

• Impact evaluation: A retrospective cohort study comparing the rate of criminal reconviction and hospital admissions for drug-involved suspects in the intervention areas (Durham, Thames Valley, West Midlands) with those in comparable areas where PDD is not available (Humberside, Hampshire, Greater Manchester).
• Process evaluation: Qualitative and quantitative research in the intervention areas to assess implementation fidelity and uncover the mechanisms by which PDD delivers its effects.
• Economic evaluation: Cost-consequence analysis to attach financial costs and savings to the services used by people in intervention and comparison areas.
• Equality assessment: Qualitative and quantitative analysis to assess systematic difference in effects according to ethnicity, gender and region.
• Realist synthesis: Bespoke work to draw out the practical and policy lessons from evaluation as a whole.

Evaluation questions

1. What are the effects of PDD schemes on offending (as measured by reconviction)?
2. What are the effects of PDD schemes on health outcomes (measured using hospital data, and drug treatment entry and retention)?
3. What other effects do PDD schemes have (e.g. on wellbeing of individuals, families and communities)?
4. Are there inequalities in the use of PDD schemes (e.g. on grounds of ethnicity, gender, levelling-up)?
5. What are the cost-consequences of PDD schemes for health, police, and other service providers?
6. How do PDD schemes produce their effects and economic impacts?

Outcome measures

Our primary outcome is:

• criminal recidivism

The secondary outcomes are:

• hospitalisation for any emergency (including acute mental health or drug and alcohol problems)
• entry in drug treatment (within 4 weeks) and retention (for at least 12 weeks)

We will also gather data on qualitative outcomes (e.g. wellbeing of communities and families).

Process evaluation, with fidelity assessment

To produce comparable information on the complex processes through which PDD schemes generate outcomes, qualitative process evaluations will be conducted by local research teams. Process evaluation design, data collection and analysis have been structured in accordance with the theory of change. It will be further informed by the findings of WP1, including the VICTORE structured descriptions and area protocol manuals. This will also produce an initial realist programme theory, which will be refined by all the other work packages.

This work will also include an assessment of the fidelity of the implementation by comparing data on the actual implementation of PDD schemes in intervention areas to the protocol manuals.

Qualitative evaluations will investigate: 

1. The implementation of PDD schemes
2. How PDD schemes produce change in drug-involved suspects
3. The impact of contexts and moderators on how PDD schemes work
4. Perceptions of stakeholders (including drug-involved suspects) of multiple outcomes (including effects on health, wellbeing, quality of life, stigmatisation, family relations, etc)
5. Fidelity of implementation of the PDD schemes

Data will be collected through semi-structured interviews and focus groups with a purposively stratified sample of service users, non-participating eligible suspects, diversion team members, police officers, diversion assessment staff, drug and other service providers, and other partner agencies. Sample size in each of strata will be determined by the achievement of data saturation.

To ascertain the fidelity of implementation to the protocols developed in WP1, we will use Carroll et al.’s conceptual framework to address the domains and components of ‘adherence’ (content, coverage, frequency, duration) and ‘moderators’ (intervention complexity, facilitating strategies, quality of delivery, participant responsiveness). 

Interview guides and focus group schedules will be developed collaboratively by academic and peer researchers on the basis of findings of WP1, following elements of the VICTORE checklist and Carroll et al.’s conceptual framework for fidelity assessment.

User Voice

User Voice will draw on its extensive experience of involving people with lived experience actively in research to recruit and train peer researchers to work alongside academic researchers in the collection and analysis of qualitative data in WP2. Peer researchers will work with academic researchers in developing the qualitative research tools, including interview and focus group schedules. They will carry out interviews and will lead focus groups with participants in the three PDD intervention areas. They will be supported by academic partners to carry out rigorous and reflective qualitative research. They will work with academic partners to document and reflect on the process of co-production of this research, leading to a research publication on this process. They will also be involved as equal partners in the writing up and dissemination of results from WP2.

Interviews

We plan for 50 interviews in each of the three areas (total = 150 across three intervention areas).

Service users (‘divertees’) (x20) and non-participating eligible suspects (x5):

• Purposive stratified sample (taking into account socioeconomic position and inequalities) – age (18-24, 25+), gender, race/ethnicity, sexuality, disability, drug types (e.g. cannabis, ‘club drugs’ and heroin/crack cocaine), socioeconomic status (e.g. education, income and occupation), criminal record, (previous) engagement with drug treatment services. The components used in this sampling method will ensure the research sample includes insights from relevant groups within the population the study is targeted at (i.e. drug-involved suspects).
• Interviews with service users will be carried out between 3 and 12 months after the diversion intervention and will explore the processes, understandings and effects of PDD.
• Interviews with non-participating eligible suspects will be carried out after they decide to not participate/not comply with the requirements of the diversion scheme.
• The interviews will focus on their understandings /views on the PDD scheme and reason for not participating/complying.
• Interviews with service users and non-participating eligible suspects will cover the following elements of the VICTORE framework and fidelity assessment: content (what PDD involves), dosage (the amount of intervention reviewed by participants (coverage, frequency duration)), responsiveness (or engagement) (acceptance by and usefulness to those receiving the intervention)), volition (the ways in which participants engage with, and respond to, diversion), rivalry (potential impact of other interventions delivered in the field).
• The majority of interviews with service users and non-participating eligible suspects will be carried out by trained User Voice peer researchers.

Practitioners and partners (x25):

• Members of the diversion team (x5) – the ‘diversion team’ will be a police-led multi-agency partnership of the various stakeholders with responsibility for implementing the PDD schemes.
• Diversion assessment staff (‘navigators’) (x5) – interviews with those who are tasked with carrying out triage assessments for diverted drug-involved suspects. Could ask for reflections on whether there is initial bounce before diminishing enthusiasm.
• Service providers (x15) – interviews with agencies providing divertees with education, treatment and/or support.
• Interviews with practitioners and partners will cover the following elements of the VICTORE framework and fidelity assessment: content (what PDD involves), dosage (the amount of intervention reviewed by participants (coverage, frequency duration)), intervention complexity, facilitating strategies (e.g. guidelines, training, monitoring and feedback), quality of delivery, responsiveness (or engagement) (acceptance by and usefulness to those receiving the intervention)), volition (the ways in which participants engage with, and respond to, diversion), context (individual characteristics, interpersonal relationships, institutional settings and infrastructure (the wider social, economic and cultural settings in which the intervention is embedded)); time (history and timing of an intervention, rivalry (potential impact of other interventions delivered in the field), emergence (how the systems under investigation continually evolve and adapt).

Focus groups

The focus groups will enable facilitated and collaborative discussion of the participants’ experience and perceptions of diversion, as well as verification and triangulation of findings from interviews. They will be facilitated by both university researchers and trained User Voice peer researchers.

Service users (x3):

• Focus groups will be designed to facilitate discussion of divertees’ understanding of diversion and experience of being diverted.
• Diversion assessment staff/service providers will assist with the recruitment of focus group participants.
• Focus groups with service users will cover the following elements of the fidelity assessment: content (what PDD involves), responsiveness (or engagement) (acceptance by and usefulness to those receiving the intervention)), volition (the ways in which participants engage with, and respond to, diversion), rivalry (potential impact of other interventions delivered in the field).

Practitioners and partners (x3):

• Towards the end of the 12-month fieldwork period, we will run separate, reflective focus groups in each implementation area with members of the following groups: Diverting police officers, assessment staff, support and treatment workers.
• These focus groups will enable facilitated and collaborative discussion of the participants’ experience and perceptions of diversion, as well as verification and triangulation of findings from interviews.
• Focus groups with practitioners and partners will cover the following elements of the VICTORE framework fidelity assessment: implementation/content (what PDD involves), dosage (the amount of intervention reviewed by participants (coverage, frequency duration)), intervention complexity, facilitating strategies (e.g. guidelines, training, monitoring and feedback), quality of delivery, responsiveness (or engagement) (acceptance by and usefulness to those receiving the intervention)), volition (the ways in which participants engage with, and respond to, diversion), context (individual characteristics, interpersonal relationships, institutional settings and infrastructure (the wider social, economic and cultural settings in which the intervention is embedded)); time (history and timing of an intervention, rivalry (potential impact of other interventions delivered in the field), emergence (how the systems under investigation continually evolve and adapt).

Quantitative data for fidelity and equity assessment 

In addition to the qualitative data gathered from interviews and focus groups, academic researchers in each intervention areas will gather anonymised quantitative data from policing partners on the implementation of PDD in their area. This will include the number of eligible suspects contacted, the number diverted, the number of divertees who attended assessment, the number who attended subsequent interventions, and the number of sessions attended, as well as the breakdown of these numbers by ethnicity and gender.

Analysis

Interviews and focus groups will be transcribed and anonymised for analysis by the local research teams (universities and User Voice), using adaptive coding in Nvivo software to identify ‘core’ and ‘satellite’ themes and nodes in the data. Quantitative data gathered from policing partners will be combined with the qualitative data for analysis of implementation fidelity.

Funding body

The Cabinet Office  -  Evaluation Accelerator Fund (EAF)