The evidence supporting applied behavior analysis is flimsy at best (if you want it to be)

Algorithmically I was exposed to Jonah Davids’ piece entitled “Are we treating autism properly? The evidence supporting behavior analysis is flimsy at best” (linked later for reasons you’ll see momentarily). Why read it? Why write about it? Why indeed. Why…

City Journal (not linked for reasons you’ll see momentarily) is an online rag published by the Manhattan Institute (not linked for reasons you’ll see momentarily) that produces mainstream Republican content with an artificial academic sheen. Reading its pages is the Olestra of complex thought; if you feel like you’re absorbing something from it, you’re just shitting yourself. For example, Wikipedia notes that throughout the years, they’ve championed:

Charter schools (public schools outperform charter schools)
”The Ferguson Effect,” wherein antagonism of police causes crime to rise (violent crime is lower today than the 1980s)
”The Broken Windows theory,” wherein “proactive” policing (e.g., police looking into small property crimes, such as broken windows) can decrease crime rates (the evidence is flimsy at best despite being introduced to the world in 1982)
Capitalism (finally, someone willing to stand up for…capital)

Why bother talking about them at all if they’re so clearly focused on politics, and not evidence? In a case of weapons-grade stopped-clock syndrome, they (Connor Harris) wrote an article that does a decent job of summarizing some evidence for Direct Instruction, phonics, and spaced responding. These are all topics covered in more depth (spaced responding, described by Ebbinghaus), and better (phonics has made the news frequently in the past several years, including an excellent podcast from NPR), in other places (Direct Instruction), but stumbling across something competent that references evidence-based practice in education is unusual. After all, the What Works Clearinghouse says that the evidence for the effects of DI is “Uncertain.” (Zig Engelmann was big mad about that) That first story, encountered some months ago, made me think that Olestra might be OK to eat, just this one time.

And if City Journal is a lab-engineered fat molecule (it definitely is), then Jonah Davids’ piece is…a wet fart? That’s actually a perfect summation of his damning conclusion:

Jonah “not too sure” Davids

The piece falls short as journalism, sure, but also as a policy statement; a good-faith exploration of the science; or even a hot-take. Let’s examine the problems from the perspective of “a person who can Google”:

Davids mischaracterizes ABA

Davids summarizes the treatment process of ABA, but it’s not sourced, because he clearly doesn’t understand how it works and has apparently never spoken to anyone in real life (IRL) about the topic.

No reward for you, Jonah

Davids mischaracterizes Early Intensive Behavioral Intervention (EIBI), wherein a hypothetical child receives 40 hours of intervention to…be rewarded for writing their name. Similarly, the writers at City Journal are paid 6 figures for…writing their names? Jonah Davids is technically two first names, so he is paid…a lot.

(Using Google Scholar, the third result gives a great summary of EIBI, including a brief description of intervention targets on page 12)

Additionally, and maybe Jonah Davids is thinking back to the good old days of bloodletting, but he characterizes ABA as

to treatment – a treatment generally considered to be established in 1959 (there were 48 US states in 1958, for reference).

Evidence for must be rigorous, evidence against can be blog posts

Inexplicably, Davids veers into terminally online territory (actually it’s because he didn’t talk to any real people for this). “Many people are saying” online that they didn’t like ABA as consumers of the service. This…is a great point! That’s a huge problem! I’m being serious: people should want treatment. If people feel mistreated, abused, hurt, or neglected, it’s an immense failing. He could write a good article about that. He didn’t, but he could.

Of course, in the absence of real data, “many people are saying” is a limited argument.

Well I’m convinced

Again, it’s a real concern, and coercion should be better understood by the average practitioner. But it’s strange to see that when building a case against ABA, suddenly the bar for evidence scrapes the ground, and we’re faced with anecdotes posted on the internet.

What’s the evidence against?

Ironically, he did not post the Cochrane review, which establishes that the research base for ABA is “low quality” – that is, there are few randomly controlled trials (RCTs) that most people are familiar with from pharmaceutical research. That’s the first result if you search “EIBI” in Google Scholar. Cochrane reviews are generally viewed as the “gold standard,” as their meta-analysis is rigorous, such as where they find that cranberry juice might not cure UTIs. Rather, he cites a researcher from Australia – without mentioning the researcher is Australian. Presumably he does this because he read (and linked, to be fair) a well-written and researched story from Spectrum News. Because, let’s face it, including the review from Andrew Whitehouse is not quite a slam dunk:

The evidence is not “very good,” it’s mostly “just ok.” I rest my case, your honor.

But it’s a good point: the greater scientific community discounts single-case design (hello again, What Works Clearinghouse!). The ABA community is aware of the issue. And here I am, making a superior argument that ABA lacks evidence. You’re welcome, Jonah.

Follow the money

It’s true that ABA, as typically practiced in 2024, consists of the following: Registered Behavior Technicians ™, supervised by Board Certified Behavior Analysts ™, providing insurance-funded services to people diagnosed with autism spectrum disorder. There is an insurance mandate in all 50 states as of 2019, ensuring that famously profligate insurers have become an unending stream of revenue to private equity companies. City Journal readers, you should both avert your eyes from the following sentence: capitalism ensures that only small numbers of people are “rich” from ABA. But still, there’s money to be made, so therefore most people will continue an unethical and destructive system (is…is Jonah Davids telling on himself?)!

It does evince a complete failure to understand the basics of ABA. As noted earlier, and even included in the surprisingly mediocre wiki page, ABA has been in use since 1959, when it was mainly in under-funded state hospitals. The Lovaas study (1987) that led to Catherine Maurice’s book (1993) that led to the in-home ABA boom was before the Behavior Analyst Certification Board was established (1998) and before the first insurance mandate was enacted in Indiana (2001). The funding cropped up decades after effective demonstration because of relentless lobbying from parent groups. It turns out Jonah Davids and I agree: healthcare should not be for-profit.

Still, a sclerotic, too-big-to-fail for-profit group that won’t examine their own evidence base – City Journal should be on guard for something like that! They should embrace their roots and tackle water fluoridation.

The ugly

This is why you never trust a man with two first names

To paraphrase the great Peter Gerhardt: when you say “low functioning,” you mean “low verbal,” because that’s what you have observed. You don’t know a person’s “functioning.” You don’t know what they understand, or know, or hear, or focus on – especially someone neurodivergent. “Lower functioning,” setting aside all controversy over assessment, is not captured by any criterion, score, or diagnosis. Yet it categorizes a person based on…well…something. Probably that something is: vocal/verbal ability.

Conclusion

Am I being too harsh on a thing sharted out into a political magazine? No.