Research digest: What do practitioners in child and adolescent mental health services think about using outcome measures?
Background:
Measuring outcomes means gaining an understanding of a patient's feelings at the start of, and at various points throughout, treatment.
As awareness of importance of measuring outcomes has grown in modern healthcare over the last three decades, their use has been recommended at policy level for child and adolescent mental health services (CAMHS). Thanks to the efforts of different initiatives and organisations, such as Children and Young People’s Improving Access to Psychological Therapies (CYP IAPT) and the Child Outcomes Research Consortium (CORC), the extent to which outcome measures are used has increased.
However, there is still a considerable gap between the recommendation at policy level and actual use of outcome measures in CAMHS, with evidence finding under a third of child and adolescent mental health services using outcome measures (Batty et al., 2013; Patterson, Matthey, & Baker, 2006). This is due to a wide range of factors, most often related to clinician’s views of outcome measures.
Research has shown that many clinicians have a positive attitude towards outcome measures, but are not convinced of their usefulness or ability to improve treatment in a clinical setting, with the majority reported to not use monitoring data that is already available to them in treatment planning (Hall et al., 2014; Garland, Kruse, & Aarons, 2003). Concern also exists as to whether the data is really helping service users, instead of simply helping managers with service evaluation. Some argue that the use of outcome measures attempts to standardise work that requires a personal approach for each service user. These concerns exist despite evidence suggesting that outcome monitoring facilitates better communication between patients, parents and clinicians, and improves treatment effectiveness.
Aim of the research:
The aim of this research was to provide an up-to-date investigation into clinician’s attitudes towards the use of outcome measures, focusing in particular on barriers to implementation and to inform the future use of outcome measures.
Method:
In order to explore these challenges, semi-structured interviews were conducted with nine CAMHS clinicians, (representing seven different job roles) from a single Mental Health Trust in South London, all of whom had been trained in outcome measures at different career stages (3 of whom used outcome measures in every session, 4 at particular points, and 2 not doing so regularly).
Findings:
Standardisation enabled use of outcome measures across the service, but was unanimously seen as a barrier for individual sessions. The standardised system helped them “use measures more consistently”, and consistent use “encouraged an evidence based approach” (pg. 4). The structured content was also seen as helpful in treatment planning and to frame discussion about the service user’s problems. On the other hand, clinicians felt standardising the use of outcome measures has potentially had a negative impact on client relationships, as they do not take into account the individual needs of each user. This was seen as potentially harmful if the service users were reluctant to use measures. Clinicians reported a key cause of potential reluctance was the language, with concerns that some users found it inaccessible or distressing.
Practical Experience of using outcome measures was seen as helpful both in individual sessions and across the service. Across the service, the use of outcome measures was seen as useful for informing commissioning decisions, although there remained “anxieties about how the data might be used” (pg.5). For individual sessions, the majority indicated that outcome measures made it easier to view progression, and discussing progress with service users was a “benefit for engaging families in care” and helps them “value [the] sessions more”. On the other hand, discussing a lack of progress was seen as an obstacle to the use of outcome measures in sessions. Largely, the clinicians felt measures were seen as a useful starting point and as indicators for future treatment, but the extent of their use should remain the choice of the clinician involved.
Resources were largely reported as a considerable barrier. In particular, “clunky [IT] systems” and “burdensome administrative processes” were highlighted as the biggest obstacles, this is in line with previous research (Valenstein et al., 2004). Clinicians were also concerned by administrative practices which ignored the clinicians in the use of outcome data, one of them voicing this as “autonomy is taken away from us” with outcome data “routinely sent out before we’ve seen [it]” (pg. 5). For individual sessions, clinicians reported completing outcomes measures could prevent talking about everything the service user wanted due to time constraints.
Training was seen as important in the use of outcome measures at both service and session levels. This provided clinicians with important knowledge and practical experience in using each particular outcome measure. Continuing support was also recommended by clinicians to sustain outcome measure use. This is in line with previous research, suggesting training in the use of outcome measures to be essential in enabling implementation both in sessions and across the service (Hall et al. 2014).
Strengths/Limitations:
The sample was recruited within one mental health Trust, with a small size, so concerns were raised that some barriers significant to this Trust may be emphasised to a greater extent, while other issues with the use of outcome measures may not manifest here. Future replications could approach more clinicians, from across more areas. It must be noted that the interviewer worked in the same Trust, this could have influenced responses and led to the withholding of some negative opinions, given the Trust is currently putting outcome measures into practice. The research, however, found that the range of positive and negative views provided suggests this to be unlikely, and felt that being interviewed by a colleague was potentially helpful in putting clinicians at ease to provide negative feedback.
Conclusions:
This research recommends training as the means to overcome the barriers presented by a standardised approach, with ongoing support crucial for dealing with the difficulties across the service involving information systems, administration processes and use of data. Future research may also be improved with the collection of data relating to the extent of actual use of outcome measures by clinicians, such as observations of practice, to allow comparisons with interview data.
References
- Batty, M. J., Moldavsky, M., Foroushani, P. S., Pass, S., Marriott, M., Sayal, K., & Hollis, C. (2013). Implementing routine outcome measures in child and adolescent mental health services: From present to future practice. Child and Adolescent Mental Health, 18, 82–87.
- Garland, A., Kruse, M., & Aarons, G. (2003). Clinicians and outcome measurement: What’s the use? The Journal of Behavioral Health Services & Research, 30, 393–405
- Hall, C. L., Moldavsky, M., Taylor, J., Sayal, K., Marriott, M., Batty, M. J., … Hollis, C. (2014). Implementation of routine outcome measurement in child and adolescent mental health services in the United Kingdom: A critical perspective. European Child and Adolescent Psychiatry, 23, 239–242.
- Patterson, P., Matthey, S., & Baker, M. (2006). Using mental health outcome measures in everyday clinical practice. Australasian Psychiatry, 14, 133–136
- Sharples, E., Qin, C., Goveas, V., Gondek, D., Deighton, J., Wolpert, M., Edbrooke-Childs, J. (2016) A qualitative exploration of attitudes towards the use of outcome measures in child and adolescent mental health services. Clinical Child Psychology and Psychiatry (2016): 1359104516652929.
- Valenstein, M., Mitchinson, A., Ronis, D. L., Alexander, J. A., Duffy, S. A., Craig, T. J., & Barry, K. L. (2004). Quality indicators and monitoring of mental health services: What do frontline providers think? American Journal of Psychiatry, 161, 146–153.