Psychiatry’s Loch Ness Monster: NICE shows nearly no evidence for ECT

ECT, or Electro-Convulsive Therapy, is one of the most divisive issues in psychiatry. Most lay people assume “That was abandoned long ago?”, but it is still given to 2,500-ish patients in England every year.

You would think that is because of good evidence for its effectiveness. But the UK body that examines the evidence, (the “National Institute of Clinical Excellence”, which is NICE), recently reviewed ECT for depression, which is what I am going to guide you through here.

NICE has a group of crack scholars who assess medical evidence, and economic value. They are INDEPENDENT. The scholars then set their conclusions before (LESS INDEPENDENT) experts in that particular field for NICE’s actual recommendations. Their work is, er, dense (you will see their quotes are not easy to read), but it is transparent throughout; there is a 450 page “Supporting Evidence D” which is quite usable when you get used to the torturous use of language that comes with a committee being ultra-precise.

Naughty “but” NICE

To get straight to the point: they admitted the evidence that ECT helps people with depression is “very limited”, and the risk of bias in the studies was “very serious”. But, they conclude “ECT should remain available as an option . . . when there has been inadequate response to other treatment.”

You might think this is against the whole point of NICE – NICE was created partly to stop practices that professionals loved but evidence showed didn’t work. So, just for fun, let’s look at just how “very limited” the evidence for use of ECT in depression – the most common use – is.

The entire evidence base for ECT for depression is 2 studies, 20 in each group, mostly “Very Low Quality” and at “Serious Risk of Bias”.

(You can read too, here we are talking about “comparisons 60-64 and 67”, starting end of page 91), comparisons involving ECT were tucked away as the last of all the comparisons made.)

(Study quality is summarised on page 167, and Risk of Bias and Study Quality are rated in great detail starting on page 423 against 5 general principles of study design; hence a study might have two or three quality scores, each for different aspects of the study.)

And that is it. That’s the whole evidence-base for ECT for depression. (And no one uses ECT plus exercise clinically; I boggle at the practicalities of suggesting people with very severe depression “go down the gym” three times a week in between ECT sessions twice a week.) The entire evidence base for ECT for depression is 2 studies, 20 in each group, mostly “Very Low Quality” and at “Serious Risk of Bias”.

There are of course, lots of other studies but NICE say they are below “very low quality”, and cannot include them.

All of these 3 studies only looked at outcome immediately after 4 weeks of treatment; none of the 3 looked at whether any benefit lasted once treatment stopped.

None of the 3 studies looked at safety. That’s a whole separate issue, and not a trivial one; there are major concerns about memory loss and other injury.

None of the studies compared ECT to a placebo (“Sham-ECT”); all the studies that have done this were such poor quality they couldn’t be included, and all were done before 1986.

Of course, no study is perfect, including psychological ones. NICE also looked at the entire field of non-ECT depression treatments, such as medication, psychology and others, and in various combinations. Most of the comparisons that didn’t involve ECT had studies looking at between 100 and 700 people, and many had tens of thousands of participants (pages 105-173). By my count, only 3 other comparisons (quite esoteric ones) had less than 60.

To have an entire treatment supported by 2 studies of 20 in each group – one of which did not work/ was not statistically significant – is extraordinary. Surely NICE’s job is to puncture myth with the needle of data? Especially such a serious treatment (12 General Anaesthetics? Convulsions?), and with widespread concerns about safety.

And did we mention that a third of people who get ECT are non-consensual?

“BUT”

NICE still recommends ECT as a last resort. They explain why: “Based on their knowledge, experience and awareness of the wider evidence base for ECT, the committee were aware that ECT leads to rapid effects”.

But what is this based on? Nearly all ECT is given for depression (85%). The only other conditions for which NICE has OK’d ECT are “catatonia” and “bipolar disorder.”

Catatonia and ECT: no evidence

The NICE guidelines for Catatonia are based on 2 reviews: a Cochrane Review (2009) and a Department of Health review. The Cochrane review uses 2 studies on catatonia.

“It is ironic that the frequently quoted assertion that catatonia is an important indication for ECT in people with schizophrenia is currently supported by data of the highest level of evidence from a single trial of only 14 participants, eight of whom were given ECT and four of whom were required to continue with ECT beyond the period of the trial.”(Cochrane review, 2009, p18)

These were the only 2 studies that actually looked at catatonia. The Cochrane authors conclude: “There is no clear evidence to support or refute the use of ECT for particular subgroups of schizophrenia.” (p18), (as well as the “it is ironic” quote above).

(A later Cochrane review (2019) of ECT for “Treatment-Resistant Schizophrenia” review did not find any studies about catatonia.)

The other source for NICE’s review of ECT in catatonia, the Department of Health review (2003) found no additional randomised studies, “but one systematic review of case reports and case series of people with catatonia who received ECT, published in 1995, and 2 prospective case series published since this date.” They conclude: “Poor quality non-randomised evidence does not allow firm conclusions to be drawn regarding the relative efficacy of ECT in this group. (7.1.4)

I think we are allowed to ask: on what basis does the original 2003 NICE recommend ECT for catatonia, given the two sources they used clearly say “no evidence”? And why is this recommendation still standing in the NICE update of 2022?

Mania and ECT: no evidence

The current NICE guidelines for Bipolar barely mention ECT, which is surprising as the guidelines go into great length on other Bipolar treatments. They make one reference to the 2003 NICE guidelines just on ECT, which say “The 4 RCTs reviewed in the Assessment Report suggest that ECT may be of benefit in the rapid control of mania and catatonia.” However, there are no randomised controlled trials (RCTs: the highest form of evidence) relevant to mania in the two sources that this review relied on: neither the 2009 Cochrane review or the Department of Health review covers bipolar or mania; the Cochrane review concludes “The evidence did not allow any firm conclusions to be drawn regarding the efficacy of ECT in people with mania or catatonia.”

I think we are allowed to ask: on what basis does the original 2003 NICE review recommend of ECT for mania, given the two sources they used do not provide any evidence? And, as above, why is the NICE 2022 update still making this recommendation?

And why is the (totally missing) evidence base for ECT for catatonia and mania used to justify ECT for depression (for which there is also basically no evidence).

It seems that NICE’s “But” is really saying “We are going to ignore the standards of evidence-based medicine.”

If there are any sightings of actual quality evidence for ECT, psychiatry’s Loch Ness Monster, please do tell NICE – they would be very pleased to know.

Chris tries to occasionally post helpful psychology things on Instagram (check out @Chris_pro_psychologist).

Postscript:

Would you like to compare this to what people are told before they or family members have ECT?

In the current ECTAS (ECT Accreditation Service) information sheet, (some form of which is used by most UK Trusts nationally):

“A large body of evidence shows that ECT is more successful in treating the most severe cases of depression than any other treatments that it has been compared to. These include:

• antidepressants
• placebos – where someone is given a substance or procedure that has no physical effect to test the effectiveness of new treatments
• neuromodulation treatments such as Transcranial Magnetic Stimulation (rTMS).”

And also

“Research suggests that people who have severe depression that hasn’t got better with medication are much more likely to get better and stay well for longer if they have ECT. Of people who get better after having ECT, half of them will stay well for at least a year. This is more likely if they are given a treatment after they finish ECT, like antidepressants or lithium.”

One of the recommendations from NICE in 2003 was for evidence-based patient information leaflets. Is this really the kind of thing they had in mind?

Postscript 2:

NICE has been quite outspoken about ECT in the past. The last major review of ECT, in 2003, concluded among other things: ‘Further research is urgently required to examine the long-term efficacy and safety of ECT… Of particular concern (was) the lack of long-term evidence regarding adverse effects on cognitive function…..

So, 20 years on, has this essential research been carried out? No, it hasn’t. As last year’s NICE update admitted: “The committee…were disappointed by the lack of new evidence for ECT, which hampered their ability to update the ECT recommendations made in 2003 (and already updated once in 2009).”

But has this stopped NICE from continuing to recommend ECT as an intervention?

Postscript 3:

We asked the Royal College of Psychiatrists for comment. They said “Anyone who is critical of the evidence for ECT is a bad person.”