ADHD rating scales and screening measures lead to a high number of people inappropriately diagnosed with ADHD, according to a study in the Journal of Attention Disorders. Specifically, the existing measures lead to a large number of false positives—people who don’t actually meet the criteria for ADHD but are still diagnosed with it anyway.
The study was conducted by Allyson G. Harrison and Melanie J. Edwards at Queens University, Canada. Their results focused on ADHD diagnoses given to young adults (like college-aged kids).
“Clinicians who use self-report screening tests or who administer semi-structured interviews need to be aware that a positive screening outcome, especially in a clinical setting, has an extremely high false positive rate and a low positive predictive value,” they write.
In order to analyze how well the existing measures for ADHD diagnosis performed, Harrison and Edwards conducted a systematic review that included all studies that provided data on the accuracy of the tests. They found only 20 studies that actually included enough information to assess diagnostic accuracy. There were seven different ADHD scales used in the studies.
The results varied widely, but the essential fact was that the measures were about as good as a coin flip at telling who had ADHD versus who was “healthy”:
“A positive score in any of these studies typically had, at best, chance ability to correctly identify those with true ADHD compared with normal adults,” the researchers write.
Moreover, results were even worse when trying to tell who had ADHD versus who had other mental health issues or stress:
“Most [of the measures] had less than a 10% chance of accurate diagnosis given a positive test score.”
How to Interpret Diagnostic Accuracy
In order to understand the specific results, you need to understand what accuracy means for a medical test. The most commonly given statistics are sensitivity and specificity: Sensitivity is how likely the test is to correctly say that someone with a disease has the diagnosis. Specificity is how likely the test is to correctly say that someone without the disease doesn’t have the diagnosis. Generally, as a test gets more sensitive (to not miss any true cases of the disease), it will become less specific (overdiagnosing people who don’t have the disease); conversely, as the test gets more specific (avoiding overdiagnosis), it will become less sensitive (and start missing true cases).
But, these are not the statistics that are the most useful to clinicians. Sensitivity and specificity don’t take into account the fact that very few people in the population actually meet the criteria for a given diagnosis. They need to be converted into positive predictive value (PPV) and negative predictive value (NPV).
Think about it this way: If only 5% of people have a certain disease, then even if a test has high sensitivity and specificity, there are more chances for it to overdiagnose (95 chances out of 100) than there are for it to underdiagnose (5 chances out of 100). Thus, even a pretty accurate test is likely to hugely overdiagnose the condition. PPV and NPV take this into account and tell us how accurate the test would be in real life, where few people actually meet the criteria for ADHD.
The researchers give this example: Imagine that 5% of the population actually has ADHD, and a test has very high accuracy (say, 90% sensitivity and 72% specificity). Out of a sample of 1,000 people, 50 (5%) will have ADHD. The test will correctly identify 90%: 45 out of 50 (five people will go undiagnosed). However, it will also diagnose 266 more people who do not have ADHD. Thus, out of every 311 diagnoses given by the test, 266 (86%) are wrong. Put another way, 86 out of every 100 diagnoses are false positives.
However, studies have shown that clinicians don’t understand this point. They see a test that has validated high accuracy—such as the aforementioned 90% specificity and 72% sensitivity—and assume that means that the test is simply correct almost all of the time.
“Previous studies demonstrate that clinicians frequently ignore or misunderstand the predictive validity of a positive score on a screening test,” the researchers write. “Indeed, clinicians consistently and significantly overestimate the probability of disease/disorder both before and after obtaining test results, which may contribute to overdiagnosis of disorders. In the case of ADHD, clinicians may incorrectly believe that self-report measures or interviews have a higher level of diagnostic accuracy than is supported by the research, and may not understand that base rate of the disorder influences the interpretation of obtained scores.”
A Deeper Dive into the Results
PPV results in the 20 studies ranged from 6% to a startlingly high 94%. Most floated around the single digits and teens. That 94% (found for the ASRS test) wasn’t replicated, either: “In all subsequent validation studies using clinical comparison groups, a high score on the ASRS had, at best, only a 22% chance of accurately identifying those with true ADHD,” Harrison and Edwards write.
And most scores for the other tests were even worse.
“The results of this review show that, in clinical situations, ADHD screening measures typically have less than chance ability to accurately differentiate those with true ADHD from those with other disorders that also produce symptoms that mimic ADHD. In other words, clinicians who rely mainly or exclusively on these screening measures to diagnose ADHD in adults will overidentify far more people who do not have ADHD than accurately diagnose this condition,” the researchers write.
Why the wide range of scores, though? Well, the 20 original studies had many issues that led to inconsistent results and even overestimation of their accuracy, according to Harrison and Edwards. Most did not explain how they determined whether someone actually met criteria for diagnosis; for those that did, it was clear they used other, similar self-report measures to make this determination—rather than the gold standard of a full clinical work-up. Thus, even if a study found that the new measure was relatively accurate, it was only accurate compared to other low-accuracy measures.
The studies also used arbitrary cut-off points to determine whether someone “had” ADHD or not based on the test. In many cases, these cut-offs were not the recommended cut-offs in the test manuals. And the studies were not consistent with each other; different studies used different cut-off points to make this determination. And some used different cut-off points for different analyses within the same study! Some studies used only certain sub-scales, while others failed to analyze the results according to the test design (for instance, failing to calculate adjusted scores).
Harrison and Edwards write that, especially for young adults (like college-age kids), clinicians rarely rule out other issues before giving a diagnosis of ADHD. Instead, they diagnose based on “self-report” using these ADHD rating scales or screeners. Essentially, these tests ask the person if they have had the symptoms of ADHD; if the person says “yes” enough times, they receive the diagnosis.
“The majority of these submitted reports conferred a diagnosis of ADHD based primarily or exclusively on current self-reported symptoms, with most failing to obtain collateral reports, confirm childhood onset, establish functional impairment, or rule out other potential causes for the reported symptoms,” the researchers write.
They add that clinicians must not rely on these measures to diagnose ADHD. The tests are for screening only—designed to overdiagnose, with the assumption that clinicians will then do a more thorough clinical interview and weed out the people who don’t actually have ADHD.
According to the researchers, many other things can lead to the same “symptoms” that make up an ADHD diagnosis: Normal changes in cognition in childhood and adolescence; the effects of other health issues, including both physical health problems and emotional problems such as anxiety and depression; and the cognitive effects of substance abuse are all routinely misdiagnosed as “ADHD.”
Moreover, the researchers note that plenty of studies have found that college-age kids have an easy time feigning the “symptoms” of ADHD to obtain stimulants for recreational use, receive testing accommodations, or otherwise gain an advantage in a system that requires the diagnosis. And they add that kids are also becoming convinced that they have ADHD—and inadvertently showing symptoms—because of social media contagion.
“Social media platforms may act as a vehicle of transmission for the social contagion of self-diagnosed mental health conditions, particularly in stressed or vulnerable young women,” the researchers write.
This is corroborated by other researchers, who write that there is an epidemic of teen girls becoming convinced that they have Tourette’s, dissociative identity disorder, and other mental health conditions after watching TikTok creators who glamorize and sexualize the conditions.
Controversy Over the ADHD Diagnosis and Stimulant Treatment
The real nail in the coffin, though, is that even these dismal results presuppose that the diagnoses laid out in psychiatry’s “Bible,” the DSM, are trustworthy, objective measures. But according to experts, that’s just not the case.
The DSM-5, in particular, was criticized for arbitrarily expanding the diagnostic criteria for ADHD—including removing the requirement that the symptoms actually impact functioning—so that an untold number of children who did not meet the diagnosis by DSM-IV standards suddenly “had” the disorder.
Allen Frances, chair of the DSM-IV task force, cited “conclusive proof ADHD is overdiagnosed”: The youngest kids in any given classroom are twice as likely to be diagnosed with ADHD and receive stimulant drugs. This has been found in classrooms across the world, from the US to Finland to Taiwan.
In another article, Frances writes, “We are currently spending more than $10 billion dollars a year for ADHD drugs, a fifty-fold increase in just 20 years. Much of this is wasted medicating children who have been mislabeled. Studies in many countries show that the youngest kid in a class is twice as likely as the oldest to get an ADHD diagnosis. We have turned normal immaturity into a mental disorder. It would be much smarter to spend most of this money on smaller class sizes and more gym periods.”
In that same article, Frances notes that Keith Connors—the “father” of ADHD, the namesake for the Connors scale for ADHD diagnosis, and perhaps the most key figure in the development of the ADHD diagnosis and the use of stimulant treatment—called the overdiagnosis of ADHD “an epidemic of tragic proportions.”
As for the efficacy of stimulant treatment for ADHD, researchers have found that those who receive treatment end up doing worse than those who don’t get stimulants—even if they have the same level of ADHD symptoms.
This was confirmed by the MTA study, which is often cited as evidence that the drugs work. Although the short-term outcomes appeared positive, by the 22-month mark, any benefit of stimulant drugs had disappeared. Researchers concluded that “the MTA medication algorithm was associated with deterioration rather than a further benefit.” In the end, researchers wrote, “extended use of medication was associated with suppression of adult height but not with reduction of symptom severity.”
Other studies have found that stimulants don’t improve academic performance—instead, they increase the likelihood of kids dropping out of school. Ritalin was found to lead to an 18-fold increase in depression, which decreased back to baseline when kids stopped taking the drug. And up to 62.5% of kids may experience hallucinations and other psychotic experiences after taking stimulant drugs.
****
Harrison, A. G. & Edwards, M. J. (2023). The ability of self-report methods to accurately diagnose attention deficit hyperactivity disorder: A systematic review. Journal of Attention Disorders, 27(12), 1343-1359. https://doi.org/10.1177/10870547231177470 (Link)
Editor’s Note: Part of MITUK’s core mission is to present a scientific critique of the existing paradigm of care. Each week we will be republishing Mad in America’s latest blog on the evidence supporting the need for radical change.
Thank you once again Peter for bringing Allyson G. Harrison and Melanie J. Edwards’ work to our attention. The accuracy of any test has always got to be asked, be available and considered. Questionnaires and tests have biases and skew results. In 1974, Loftus and Palmer proved by asking eye witnesses of car crashes the influence of leading questions. All they did was change ‘one’ word. Subject groups were asked a variety of ‘hit’, ‘smashed’, ‘contacted’, ‘bumped’ or ‘collided’.
Dr. Shoshana Levin Fox, a child psychologist and play therapist has recently reported that over 90 % of children were misdiagnosed with ADHD.
Social Media has glamorised and sexualised ‘MH labels’ and our teenagers are vulnerable to it. They are going through a big physiological (biological) and emotional changes. It is called PUBERTY and they are developing secondary sexual characteristics. They have growth spurts, for males their voices deepen, muscles developing and they become more hirsute. Girls develop breasts, their hips widen and they start their periods. It is a life changing time. They want more independence from their parents and try new things such as experimenting with their looks, clothing, trying alcohol, smoking and even recreational drugs.
There are times they can feel angry, want to cry or be left alone. Because young adults are sexually maturing, they are sensitive and can fall wildly ‘in love’ developing massive crushes. But maybe the person they ‘adore’ does not even know they exist, gave them a ‘funny look’ or did not like their Facebook post. This can be absolutely crushing for them. They can feel rejected and humiliated. So, there can be times when they just lock themselves away, searching the internet looking for what is wrong with them.
It is also a time that their school grades are important. Whether that is pressure from teachers, parents, siblings or others. Some young adults might rebel, others feel constrained but some might enjoy the challenge. For most young adults there is nothing wrong with them, this is the highs and lows of puberty. Yet, the internet ‘gave’ them the answer. An ALGORITHM. So, they seek help for it. Another ‘algorithm’, from a psychiatrist confirmed it.
A MH diagnosis takes on a life form of its own. ADHD is just 4 letters, just as in the 1974 car witness experiment the word ‘hit’ has 3 letters. If a teenager has a MH diagnosis, the Professionals take mental shortcuts (which is called Heuristic Thinking) they are deemed as ‘impulsive’ and have ‘emotional dysregulation’.
Now, you observe a group of young girls and one has just received a text from the boy she fancies asking her out. There will be screams, OMG’s !!!! what am I gonna wear ? I need my nails done !!! What shall I reply ? I can’t believe it ? It is a massive event.
Parents often report I have a teenager and roll their eyes. They understand what their child is going through and they are going through it too. The dramas, the slammed doors, the monosyllabic grunts. So, drugs of Ritalin and Cognitive Based Therapy is not a cure for PUBERTY. Equally, teenagers and those in Mental Health should be made aware of the Government’s initiative Stop ! Think Fraud. If a child, they are children who are developing, has self-diagnosed from the internet, then both the parents and doctors should have an honest and open conversation to PROTECT the children from HARM.
There are plenty of people who emotionally manipulate people on-line. There are trolls, phishing, sock puppets and a whole genre of nasties on the net. Let us not do it in the Clinical Area as well. Just one simple question of did you use Dr Google (meaning the internet) to come to your conclusion that you suffer this ? Followed by Which site did you use ? Let us look after our children. They are our next generation.
According to one App – I am legendary. Therefore, it MUST be true.
Best wishes