Learning from a Learning Collaboration: The CORC Approach to Combining Research, Evaluation and Practice in Child Mental Health
Abstract: This paper outlines the experience of the Child Outcomes Research Consortium—formerly known as the CAMHS Outcomes Research Consortium; the named changed in 2014 in recognition of the widening scope of the work of the collaboration; a learning collaboration of service providers, funders, service user groups and researchers across the UK and beyond, jointly committed to collecting and using routinely collected outcome data to improve and enhance service provision and improve understanding of how best to help young people with mental health issues and their families.
Fleming, I., Jones, M., Bradley, J., & Wolpert, M. (2014). Learning from a learning collaboration: The CORC approach to combining research, evaluation and practice in child mental health. Administration and Policy in Mental Health and Mental Health Services Research, 1-5.
From ‘reckless’ to ‘mindful’ in the use of outcome data to inform service-level performance management: perspectives from child mental health
Abstract: In the recent report on patient safety in the National Health Service (NHS) in England, Don Berwick calls on the NHS to align the necessity for increased ‘accountability’ with the necessity to ‘abandon blame as a tool’ in order to develop a ‘transparent learning culture’.1 Sir Bruce Keogh, Medical Director NHS, and colleagues’ recent analysis of outlier hospitals based on mortality data marks a key step on this journey, but has led to high-profile debate about the risk of possible ‘reckless’ (Sir Bruce Keogh's term) use of data if appropriate parameters are not established.2 ,3 If these and other equivalent proxies for outcomes are to be used safely and effectively to support performance management and quality improvement in the ways envisioned by both Keogh and Berwick, it is crucial to establish clearly agreed operational procedures. Drawing on our experience of collecting and interpreting outcome data in the challenging context of child mental health across the UK, we suggest adoption of a MINDFUL framework.
Wolpert, M., Deighton, J., De Francesco, D., Martin, P., Fonagy, P., & Ford, T. (2014). From ‘reckless’ to ‘mindful’in the use of outcome data to inform service-level performance management: perspectives from child mental health. BMJ quality & safety, bmjqs-2013.
Considering Harm and Safety in Youth Mental Health: A Call for Attention and Action
Abstract: The possibility of harm from mental health provision, and in particular harm from youth mental health provision, has been largely overlooked. We contend that if we continue to assume youth mental health services can do no harm, and all that is needed is more services, we continue to risk the possibility that the safety of children and young people is unintentionally compromised. We propose a three level framework for considering harm from youth mental health provision (1. ineffective engagement, 2. ineffective practice and 3. adverse events) and suggest how this framework could be used to support quality improvement in services.
Wolpert, M., Deighton, J., Fleming, I., & Lachman, P. (2014). Considering Harm and Safety in Youth Mental Health: A Call for Attention and Action. Administration and Policy in Mental Health and Mental Health Services Research, 1-4.
Measurement Issues: Review of four patient reported outcome measures: SDQ, RCADS, C/ORS and GBO – their strengths and limitations for clinical use and service evaluation
Background: There is an international drive for routine use of Patient Reported Outcome Measures (PROMs) across all health services including in relation to Child and Adolescent Mental Health Services (CAMHS). A number of reviews have summarized the validity and reliability of well‐being and mental health measures for children but there are fewer attempts to consider utility for routine use.
Wolpert, M., Cheng, H., & Deighton, J. (2014). Measurement Issues: Review of four patient reported outcome measures: SDQ, RCADS, C/ORS and GBO–their strengths and limitations for clinical use and service evaluation. Child and Adolescent Mental Health. Doi: 10.1111/camh.12065
Comparison of indices of clinically meaningful change in child and adolescent mental health services: difference scores, reliable change, crossing clinical thresholds and ‘added value’ – an exploration using parent rated scores on the SDQ
Background: Establishing what constitutes clinically significant change is important both for reviewing the function of services and for reflecting on individual clinical practice. A range of methods for assessing change exist, but it remains unclear which are best to use and under which circumstances.
Wolpert, M., Görzig, A., Deighton, J., Fugard, A, J, B., Newman, R. & Ford, T. (2014) Comparison of indices of clinically meaningful change in Child and Adolescent Mental Health Services (CAMHS): difference scores, reliable change, crossing clinical thresholds and “added value”; an exploration using parent rated scores on the SDQ. Child and Adolescent Mental Health.
What kinds of goals do children and young people set for themselves in therapy? Developing a goals framework using CORC data
Abstract: Agreement on goals is thought to be central in successfully building a good therapeutic alliance which in turn improves outcomes. The qualitative idiosyncratic nature of goals set by children and young people in therapy has been relatively unexplored. We investigated service users’ account of the goals they set for them- selves in UK child and adolescent mental health services (CAMHS) using data collated by CAMHS Out- comes Research Consortium (CORC) members. Six services supplied their goals data for analysis: NHS targeted, specialist, and highly specialist services; a modality specific professional body; and a therapeutic community. Service users were 80 children and young people who had visited the services between 2007 and 2011. Emerging themes arising from key aspects of 241 goals from the participants (as agreed with and recorded by the clinician) were analysed using thematic analysis. The resulting taxonomy of goals consisted of three overarching themes and 25 lower level categories. Inter-rater reliability between two researchers was substantial for major themes (Cohen’s kappas from 0.78 to 0.88 and statistically significantly above 0.6) and 14 of the 25 subthemes had substantial agreement. The top five goals mainly concerned personal growth, functioning, and coping with specific symptoms and problems. We discuss these results and differences with a large analysis done in the USA on the inverse ‘top problems’.
Bradley, J., Murphy, S., Fugard, A. J., Nolas, S. M., & Law, D. (2013). What kinds of goals do children and young people set for themselves in therapy? Developing a goals framework using CORC data. Child & Family Clinical Psychology Review, 1(1), 8-18.
Satisfaction in Child and Adolescent Mental Health Services: Translating Users’ Feedback into Measurement
Abstract: The present research addressed gaps in our current understanding of validity and quality of measurement provided by patient reported experience measures. We established the psychometric properties of a freely available experience of service questionnaire (ESQ), based on responses from 7,067 families of patients across 41 UK providers of child and adolescent mental health services, using the two-level latent trait modeling. Responses to the ESQ were subject to strong ‘halo’ effects, which were thought to represent the overall positive or negative affect towards one’s treatment. Two strongly related constructs measured by the ESQ were interpreted as specific aspects of global satisfaction, namely satisfaction with care, and with environment. The Care construct was sensitive to differences between less satisfied patients, facilitating individual and service-level problem evaluation. The effects of nesting within service providers were strong, with parental reports being the most reliable source of data for the between-provider comparisons. We provide a scoring protocol for converting the hand-scored ESQ to the model-based population-referenced scores with supplied standard errors, which can be used for benchmarking services as well as individual evaluations.
Brown, A., Ford, T., Deighton, J., & Wolpert, M. (2012). Satisfaction in child and adolescent mental health services: Translating users’ feedback into measurement. Administration and Policy in Mental Health and Mental Health Services Research, 1-13.
Routine outcomes monitoring as part of children and young people's Improving Access to Psychological Therapies (CYP IAPT) – improving care or unhelpful burden?
Abstract: This brief commentary article considers the implications of intensive outcome monitoring which is central to children and young people's Improving Access to Psychological Therapies (CYP IAPT) in England and Wales. Services are being provided with a range of free software solutions to enable data collection, and guidance on interpretation of the measures, but there will still be some burden of data entry and collation for already overstretched services. It may be that the utility of the feedback will go some way to offset the sense of burden but this remains to be seen. Whether commissioners and others will rise to the challenge of supporting this aspect may prove crucial to the success or otherwise of such intensive ROM use. Many aspects of the CYP IAPT approach are new and whilst drawing on experience from earlier pilots of session by session monitoring in CAMHS both in the UK and abroad, and from Adult IAPT, there is likely to be much for us to learn. Time will tell whether the approach helps to improve the care children and young people receive but we are cautiously optimistic.
Wolpert, M., Fugard, A.J.B., Deighton, J., & Gorzig, A. (2012). Routine outcomes monitoring as part of children and young people's Improving Access to Psychological Therapies (CYP IAPT) - improving care or unhelpful burden? Child and Adolescent Mental Health, 17(3), 129-30
Patient-reported outcomes in child and adolescent mental health services (CAMHS): Use of idiographic and standardized measures
Abstract: There is increasing emphasis on use of patient-reported outcome measures (PROMs) in mental health but little research on the best approach, especially where there are multiple perspectives.
Wolpert, M., Ford, T., Trustam, E., Law, D., Deighton, J., Flannery, H., & Fugard, A.J.B. (2012). Patient-reported outcomes in child and adolescent mental health services (CAMHS): use of idiographic and standardized measures. Journal of Mental Health, 21(2), 165-173.
CORC's Survey of Routine Outcome Monitoring and National CAMHS Dataset Developments: A Response to Johnston and Gower
Abstract: The CAMHS Outcome Research Consortium has 44 members across Great Britain who are collaborating to introduce routine outcome monitoring into clinical practice. We discuss the constraints brought up by our members and by Johnston and Gowers, as well as recent work in relation to the development of a national CAMHS dataset.
Ford, T., Tingay, K., & Wolpert, M. (2006). CORC’s survey of routine outcome monitoring and national CAMHS dataset developments: A response to Johnston and Gower. Child and Adolescent Mental Health, 11(1), 50-52.