In this blog, Stephen Kellett and Mel Simmonds-Buckley describe their 2021 meta-analysis of outcome research into cognitive analytic therapy.
This project was an attempt to review all of the outcome studies completed to date looking at clinical outcomes for CAT. Some studies were excluded because they were case studies or the authors had not included information we could use. In order to complete the project, we looked at CAT’s effectiveness on three outcomes (depression, interpersonal problems and general functioning) looked at how CAT compared to other treatments/therapies, and assessed how acceptable CAT was (i.e.what the dropout rate was).
Both the people that receive psychotherapy, and NHS commissioners, need to know that psychotherapies chosen and provided are acceptable, effective and efficacious. What we mean by these terms is:
- Effectiveness: that clients’ symptoms reduce over treatment time.
- Efficacy: that the therapy improves a client’s mental health compared to doing something else, such as another psychological treatment or medication. These are often the ‘control’ groups in randomised controlled trials. Researchers randomly allocate participants either to CAT or another approach, and then see what is most useful.
- Acceptability: that clients continue to turn up for the therapy and stay in treatment until it’s completed.
What we wanted to achieve with this project was a ‘meta-analysis’ of the CAT research studies that have looked at effectiveness and efficacy. This was in order to arrive at some summary statements about whether CAT works.
What we did
Carrying out this meta-analysis meant finding all the relevant studies as a first step. We did this using what is called a systematic review. Then we rated the quality of these studies according to a set of standards or criteria. Next, we created a spreadsheet of outcomes and also listed the features of each study.
Once we had all the information in a spreadsheet, we did a narrative synthesis of the features of the studies. We also applied statistical techniques to combine the findings from all the studies. This produced a more accurate estimate of what is likely to be the ‘true’ effect of CAT.
We combined the studies by ‘weighting’ the findings in each of them. For example we can have more confidence in the results of larger studies on the impact of CAT or when more precise studies showed where CAT had more influence on the overall treatment effect.
However we could not combine the studies in this way when looking at the evidence on acceptability. Instead we worked out the average ‘dropout’ rate for CAT as a first step. This took into account the proportion of clients who left therapy before completing it.
We have subsequently completed a proportional meta-analysis on CAT treatment refusal rates (i.e. those who are screened and allocated to CAT, but then do not attend) and treatment dropout rates (i.e. those who start CAT and drop out before finishing the 8, 6, or 24 session contract). We will share those results when the paper has been accepted.
So, in this current effectiveness meta-analysis, we wanted to explore the impact of CAT on three areas:
- depression (i.e. how down or low in mood people feel)
- global functioning (i.e. how people are psychologically getting on generally in their lives), and
- interpersonal problems (i.e. how well people’s relationships are going)
So, what did we find?
There were twenty-eight studies and the quality of the papers did vary. When we considered the quality of the methodologies in the original studies used, then overall the study quality was poor. This was because clinicians in routine services have collected a lot of the evidence on CAT. In such settings, the methodologies used mean the results are pretty generalisable, but not necessarily that scientifically well-controlled. Eighteen of the studies (64%) tracked changes over time with no comparison or control group. The remaining ten studies (36%) were randomised control trials (RCTs). These compared CAT to a comparison or control group over time, and so were better controlled from a scientific point of view.
In the meta-analysis we could use twenty-five studies. The studies we looked at showed that CAT is offered to people with complex problems such as long-standing and complex trauma . Therapists tended to deliver it on a one-to-one basis in public health settings, typically in the NHS. There were however, good CAT studies from other countries, such as Australia.
Effectiveness
At the end of treatment people who had CAT tend to have significantly reduced depression, better interpersonal functioning and reduced global distress. The effect was larger for global distress and reduced depression than the improvement in interpersonal functioning. It didn’t make a difference whether the clients were male or female, or how old they were.
It did not matter in terms of the global functioning or depression outcomes whether the CAT therapist was still in training, or was qualified. At first, we saw larger effects for improved interpersonal functioning when qualified CAT therapists provided the therapy. However, once we corrected the analysis for all the analyses that were being made at the time, this relationship stopped being significant in statistical terms .
In the studies that followed clients up after therapy finished, clients had held onto the gains that they had made during CAT. The follow up periods were normally at around 6-months after the end of therapy. At follow-up, there was also a significant improvement in clients’ interpersonal difficulties. This means that the effectiveness of CAT is long-standing and the interpersonal focus of CAT seems to do its work over time.
Efficacy
CAT was compared to controls in nine randomised controlled trials (RCTs) because one of trials had compared one version of CAT with another and so we could not use that as a treatment comparison. In the other studies, CAT was typically compared against ‘treatment as usual’ (i.e. what the patient would normally receive in the service). In some cases, the comparison was with being on a waiting list, or having another psychotherapy. When we compared outcomes between people in the CAT and control groups, we found CAT was slightly (but significantly) more effective at the end of treatment. However this difference had disappeared by follow-up. This shows that CAT has treatment efficacy, at least in the short-term.
Acceptability
The dropout rate for CAT overall ranged between 0-39% of the sample. The average dropout rate was 16% across all the CAT outcome studies. During CAT trials specifically, the average dropout rate was higher, at 23%, and this compared to 26% in the controls.
You can find the original study here:
https://bpspsychub.onlinelibrary.wiley.com/doi/10.1111/papt.12286
Stephen Kellett & Mel Simmonds-Buckley
Dr Stephen Kellett is a Consultant Clinical Psychologist & CAT Psychotherapist. He works in an inpatient setting in the NHS and contributes to ACAT-accredited training programmes. He is also an Honorary Senior Lecturer at the University of Sheffield’s Clinical & Applied Psychology Unit and a member of the PEARLS Research Lab.
Dr Mel Simmonds-Buckley is a Lecturer with the University of Sheffield’s Clinical & Applied Psychology Unit. She is also a member of the PEARLS Research Lab, and a researcher in an NHS Trust
https://pearlsresearchlab.group.shef.ac.uk
This updated version (June 2022) replaces the original article published in November 2020. It includes more detail about the statistical analyses, and conclusions drawn from these, in response to viewer feedback.
If you would like to see a listing of recent research into CAT, visit our page on the CAT Evidence Base.
So, How Effective is Cognitive Analytic Therapy? by Dr Stephen Kellett & Dr Mel Simmonds-Buckley is licensed under CC BY-SA 4.0