Exploring a new outcome measure for single-session work developed by Kooth (a web-based therapy service)
CORC members, Kooth, have been providing online single-session counselling support to young people since 2004. The single-session approach aims to address young people’s immediate concerns in one session (although professionals might signpost to additional, longer-term support). Kooth found that there was a lack of outcome measures specifically developed to track the impact of this type of work, which uses a strength-based approach to help young people focus on in-session goals rather than longer-term therapeutic goals and outcomes.
Kooth set out to fill the gap between the short-term nature of single-session support and the longer-term measurement of mental health difficulties. In order to measure the impact of the single-session support they provide, Kooth developed the Session Wants And Needs Outcome Measure (SWAN-OM). The aim of the measure is to help shape the content of single-session support sessions, as well as to track important areas of impact, as indicated by young people immediately after the session. This impact tracking immediately afterwards is particularly important to capture in scenarios where a longer-term follow-up with the young person might not be possible (e.g., some support might be provided anonymously).
The SWAN-OM items were developed using the Kooth theory of change model, and were based on goals that were set by young people accessing Kooth and analysis of text-based support transcripts. From there, workshops were held with professionals and with young people, to refine the items. Coproduction in measure development is incredibly important, particularly given that research has shown that the outcomes that are of key importance for practitioners, and that are often those reflected in outcome measurement and service delivery, are not necessarily the outcomes that are of key importance for young people (Krause et al., 2019). This development stage of the SWAN-OM is detailed in Kooth’s publication here: de Ossorno et al. (2021).
Once they had the first version of SWAN-OM, Kooth asked us to collaborate with them to explore the validity of the measure. We were interested in the psychometric properties of this measure because we know that many of the children supported by our other CORC members are also seen for a single session (Edbrooke-Childs et al., 2021), and that there was a gap in measurement. We split the project into two phases. In phase one, we explored the use of the SWAN-OM in Kooth’s online text-based chat support environment. As comparators, we used pre-existing, already-researched measures that asked about the experience and impact of the session and changes in positive and negative affect – constructs SWAN-OM also aimed to measure.
This project was interesting to us because the structure of the SWAN-OM meant that traditional psychometric testing was not the most appropriate way to fully explore how the measure performs and is responded to. This is because young people are not required to select and rate all SWAN-OM items, creating the appearance of a high rate of ‘missing data’, associated with the items not selected, and the way any two measures are completed is unlikely to be the same. This meant that we were able to think more dynamically, and consider the measure at an item level, rather than the traditional subscale – or total – level. Young people only score the measure at time 2 as well, which counted out any analysis we would have usually planned that explores change over time. Instead, we considered a single data time point of impact.
We found that most young people accessing single-session support from Kooth completed the SWAN-OM at both time points, which suggested high acceptability for the measure. We also found that most commonly, young people selected items specifying their goal for the single session was ‘to feel better’ or ‘find ways I can help myself’. This suggests that young people might access single-session support as a means to help them immediately feel better, but also to help them think about ways to help themselves, perhaps for signposting to other sources of support. This does fit with the initial aims of single-session support as outlined. We also found that scores on the SWAN-OM items were highly correlated with items on the comparator measures, which suggests that the measure has good concurrent validity – or that it measures what it intends to measure. There were, however, a few items that young people tended to not select very much, or at all, and we made suggestions for refinements to the measure based on this and other findings relating to the initial structure of the measure.
Kooth colleagues refined the measure, and the SWAN-OM version 2.0 was taken forward into phase 2 of the validation study. We explored the use of the refined measure in some face-to-face as well as online settings, and there was some additional work to explore young people’s and practitioners’ experiences of completing the measure, as well as thinking about what a meaningful score on the measure might be. We reported our findings to Kooth towards the end of 2022, so watch this space!
We are interested to see where our colleagues at Kooth take this measure next. That might involve some more development or testing work. We do know that it would be great for there to be a tool that could be used to help young people and practitioners explore and track single-session work.
For more information about the measure, please get in touch with our Kooth colleagues at contact@kooth.com .
Reference:
de Ossorno Garcia, S., Childs, J., Salhi, L., Ruby, F. J., Sefi, A., & Jacob, J.
Examining concurrent validity and item selection of the Wants and Needs Outcome Measure (SWAN-OM) in a Children and Young People web-based therapy service. Frontiers in Psychiatry, 14, 89. https://doi.org/10.3389/fpsyt.2023.1067378
de Ossorno Garcia, S., Salhi, L., Sefi, A., & Hanley, T. (2021).
The Session Wants and Need Outcome Measure: The Development of a Brief Outcome Measure for Single-Sessions of Web-Based Support. Frontiers in Psychology, 12, 748145. https://doi.org/10.3389/fpsyg.2021.748145
Edbrooke-Childs, J., Hayes, D., Lane, R., Liverpool, S., Jacob, J., & Deighton, J. (2021).
Association between single session service attendance and clinical characteristics in administrative data. Clinical Child Psychology and Psychiatry, 26(3), 770-782. https://doi.org/10.1177/13591045211002
Krause, K. R., Bear, H. A., Edbrooke-Childs, J., & Wolpert, M. (2019).
What outcomes count? Outcomes measured for adolescent depression between 2007 and 2017. Journal of the American Academy of Child & Adolescent Psychiatry, 58(1), 61-71. https://doi.org/10.1016/j.jaac.2018.07.893