The IAPT Service Is an Abject Failure

In 2012, an editorial in the prestigious journal Nature claimed that the UK’s IAPT Service is “world-beating”—meaning that the service is the world’s best for treating mental health concerns. Now that 10 years have passed, it’s clear that this was not true. Despite the hype, the IAPT is hardly “world-beating.” In fact, it is a doubtful model for other countries to follow.

The IAPT (“Improving Access to Psychological Therapies”) was created in 2008 and fully funded by the UK government (though restricted to provision only in England). The program claims to provide evidence-based psychological treatment, most commonly in the form of cognitive behavioural therapy. It was initially focused on patients with depression and anxiety disorder diagnoses, but its scope has been gradually expanded to include the psychological treatment of long-term physical conditions such as chronic fatigue syndrome.

Patients either self-refer (at no cost) or are referred to the IAPT by their primary care physician/general practitioner. Clients are initially assessed during a 30-minute telephone conversation by a Psychological Wellbeing Practitioner (PWP) who is not an accredited therapist and is not trained to diagnose. IAPT clients initially receive low-intensity CBT (six hours or less of therapy contact) delivered by a PWP. If they are deemed not to have responded to this, on the basis of a psychometric test, they can be “stepped up” to high-intensity CBT delivered by an accredited psychological therapist.

How Well Is the IAPT Doing?

Proponents of the IAPT cite its outcomes for the clients who complete their treatment sessions—but over half of IAPT clients don’t even attend two sessions. The response of the IAPT to this massive disengagement with the service—clients “voting with their feet,” as it were—is currently to offer training to its staff to “streamline” matters by encouraging them to use a computer algorithm to decide which clients go where. This training, which was held on June 22, 2022, involves expert advice from the CEO of Limbic, an artificial intelligence company. Thus, not only are IAPT staff to be distracted by clients having to complete psychometric tests at each encounter but now additionally by the administration of the computer algorithm.

Yet Limbic claims it is “an AI assistant for clinical assessments in IAPT—improving access, reducing costs and freeing up staff time.” Where is the independent evidence for this? What clients really want is to be listened to, but there is little chance of this any time soon. Conquering disengagement in this manner is a forlorn hope.

IAPT’s 2019 manual states that IAPT staff do not make a diagnosis. However, the treatment recommendations of NICE (the UK’s National Institute for Health and Clinical Excellence) are explicitly tied to a taxonomy of specific treatments for identified disorders (largely, diagnosis-specific variations on CBT). IAPT legitimises itself by a claim to be NICE compliant, but how could they be if they don’t make diagnoses? How can they follow NICE’s specific treatment guidelines without first assessing which condition they are treating?

Whilst the IAPT’s claim to be NICE-compliant might be music to the ears of NHS England and clinical commissioning groups, it is simply not true. No independent fidelity checks have ever have been made on IAPT’s ministrations. IAPT allegedly provides specific CBT protocols for different disorders, but it has provided no evidence that it does this.

In my examination of the physician’s records of 90 IAPT clients, I found no evidence of compliance with CBT protocols. IAPT operates simply as a business with superb marketing.

In no independent audit of IAPT, have clients been asked “are you back to your usual self since this treatment?” This is a goal that is meaningful to the client—with the possible supplementary question, “for how long have you been back to your usual self?” In papers published by IAPT staff, no such real-world outcome measures have ever been used. Instead, reliance has been placed on changes in psychometric test scores. Yet without a control group, it’s impossible to know whether these changes would have occurred with the passage of time anyway. After all, people present to psychological treatment at their worst, and at any other time, they are likely to score better. In statistics, this principle is known as regression to the mean, and it is vital to account for this effect.

Further, the client knows his/her therapist sees their completed questionnaire, pushing them to respond more positively. A client may also not want to endorse negative responses on the questionnaire because it would mean acknowledging that they have wasted their time in therapy. These are known issues with self-report measures that are commonly cited as important limitations in research studies.

Finally, the questionnaire results also give no indication of whether any improvement has lasted for a period that the client would regard as meaningful, such as eight weeks or (hopefully) more.

In an era of personalised medicine, it is essential that the voice of the client is heard. It is an almost ubiquitous failure of studies to employ a primary outcome measure that a client can identify with—known as patient-centred outcomes. Perhaps the nearest proxy is loss of diagnostic status (full recovery) determined by an independent assessor. The burden of proof rests on providers of services to demonstrate that their treatments are making an important difference to a client’s life in a way that is clearly recognisable to the client. IAPT has not cleared this evidential bar.

The use of an algorithm is not incompatible with a clinician operating on a knowledge base of what works for whom. But it has to be first demonstrated that the former constitutes added value over the latter. The burden of proof is on AI advocates to demonstrate its relevance in a particular context. This cannot be done by a company with a vested interest in AI.

It is perfectly possible that using two systems, an algorithm and clinical judgement might create confusion. Should an IAPT client not fare well (for example, if a client dies by suicide) when treated under such a dual system, could IAPT convince a court, on the balance of probability (the UK benchmark in personal injury cases), that its approach was evidence-based? After all, an AI approach is not something that NICE has recommended—so it flies in the face of the guidelines. Traditionally, courts are swayed by eminence-based evidence (such as the word of authority figures like psychiatrists), but they have become more aware (at least in the UK) of the perils of unbridled clinical judgement.

Inadequacies of IAPT’s Chosen Metric

IAPT’s chosen metric is a change in score on two psychometric tests. Clients are deemed to have responded if on the PHQ-9 (a measure of the severity of depression) their test score has gone from above 10 to below 10 with treatment, with a difference of at least 6 points, and if on the GAD 7 (a measure of the severity of generalised anxiety disorder) their score has gone from above 8 to below 8 with treatment, with a difference of at least 4 points. This approach has numerous deficiencies:

• It is not known whether the completer of these two measures was actually suffering from depression or generalised anxiety disorder, because IAPT clinicians do not make diagnoses. Further, it is not known whether either of these “conditions” was the disorder that was causing the most impairment in functioning.
• When test results are observable by the treating clinician, clients may wish to be polite and relay a lower score, particularly as they do not want to feel that they have wasted their time in therapy.
• No attention control comparator: Changes in test scores are meaningless unless there is a comparator group that has received the same attention and a credible rationale. It’s unclear whether the therapy resulted in the improvement, or if the client would have improved with a “placebo” therapy (simply receiving professional “attention” without CBT techniques).
• Regression to the mean: Clients typically present at their worst, so with the mere passage of time they will score more modestly.
• The vagaries of a minimally important difference (MID) in a score: MIDS are established by statistical comparisons between an “ill” and “well” group, but they do not necessarily indicate the clinically important differences. They also fail to address the client’s perception and there is no personalisation of treatment outcome. Clients are given no voice.

Haemorrhaging

Last year about a third of referrals (30.1%) to the IAPT service did not access it. Of those who accessed the service over a third (38%) had just one treatment session. Thus, by the start of the IAPT race over half of people (57%) have not engaged in treatment (as defined by IAPT’s metric of attending two or more sessions). If these results applied to a physiotherapy service it would raise serious doubts about the suitability of the service! Why then does the UK National Audit Office studiously avoid scrutiny of IAPT, which costs the taxpayer over £1 billion a year?

Sub-Therapeutic Dose and Treatment Outcomes

Of those who engaged in IAPT, the average number of sessions was 7.5, which is well below the 12 to 20 sessions that NICE recommends for depression and anxiety disorders (IAPT’s primary targets). Are we to believe that IAPT’s clinicians are so skilled that they can achieve recovery with only half the dosage of therapy as that delivered by therapists in the randomised controlled trials of CBT for depression and the anxiety disorders?

The benchmark set by trials of therapy as a treatment for these conditions is that at least 50% must “recover” (no longer meet the criteria for a diagnosis). But there is no evidence that IAPT has achieved this. My own research suggests that only the tip of the iceberg recover.

In my capacity as an Expert Witness to the Court, I examined 90 IAPT clients who had been through the service either before or after the personal injury. Only 9.2% of subjects lost their diagnostic status, as assessed using the “gold standard” SCID interview. It mattered not whether subjects were assessed before or after their personal injury. (These findings are in need of replication by clinicians independent of IAPT with a non-litigant population.)

Diagnostic Creep

IAPT appears not to so much follow the data, but to follow funding opportunities. It has branched out into territory where angels fear to tread: Medically Unexplained Symptoms (MUS). MUS is an umbrella term embracing conditions as diverse as chronic fatigue syndrome, fibromyalgia, and irritable bowel syndrome. IAPT’s contention is that exaggerated negative beliefs about symptoms, and maladaptive behaviours (e.g., avoidance), play a pivotal role in the maintenance of symptoms in these conditions. The therapeutic task, as they see it, is to then to modify these cognitions and behaviours.

What happens if a client protests that they have valid medical symptoms? Well, IAPT clinicians are taught not to openly disagree and not to say “it is all in your mind.” But to nevertheless continue to focus on the “exaggerations in beliefs and avoidance behaviours”—so much for honesty.

Along with Keith Geraghty from the University of Manchester, I have published a critique of this sojourn. We identified a series of seven core problems and failings of the IAPT, including an unproven treatment rationale, a weak and contested evidence base, biases in treatment promotion, exaggeration of recovery claims, under-reporting of drop-out rates, and a significant risk of misdiagnosis and inappropriate treatment. We concluded that psychotherapy should not become the default option when patients have “medically unexplained symptoms.”

The term “medically unexplained symptoms” should be written in lowercase to avoid conveying the impression that a meaningful homogenous entity is being described. The term should not enter the taxonomy of disorders. Importantly, even the DSM-5 has shunned recourse to the term. But unfortunately, IAPT only pays lip service to the standard diagnostic criteria.

Big Pharma and IAPT

Whilst some large pharmaceutical companies have greatly served the public good with regard to COVID, their performance with regard to psychotropic drugs is much less impressive. Researchers like Read and Moncrieff pin their hopes on the efficacy of CBT demonstrated in randomised controlled trials, but make no mention that CBT, as delivered in routine practice, bears little relationship to the protocols utilised in RCTs. Just as the claims of pharmaceutical companies require critical appraisal so too does the UK IAPT service.

In an article in the British Journal of Clinical Psychology, I explained what the IAPT needs to do to get back on track. I noted that in their published papers, IAPT staff do not declare their allegiances and indeed state that they have no conflict of interest! This dishonesty needs to change. I suggested that there is a need for IAPT to prove their case using the standards of proof that are required in other areas of medicine.  To this effect, they cannot simply claim that they deliver evidence-based treatment—there has to be documented evidence for it that would be persuasive to an independent observer.  The IAPT service has to demonstrate that it provides an added value over someone attending an advice centre or counselling service. Yet the IAPT has yet to demonstrate that it makes an important enough difference to clients’ lives that they would recognise it.

This catalogue of omissions may create a sense of déjà vu for those who have followed the machinations of Big Pharma with regard to mental health. Unfortunately, there has only been a deafening silence from the IAPT regarding my critique.

Editor’s Note: This post was originally published on Mad in America in June 2022 and is shared here with permission.