Mark Blaxill: Lies, Damned Lies and CDC Autism Statistics

That might sound a bit harsh to the disinterested observer. After all, didn’t the CDC just come out and admit that the rates have gone up and are basically 1 in 100? One might think it would be unfair to criticize the new administration for publishing numbers that essentially confirm what the autism community has been arguing for years. How exactly, one might reasonably ask, could one argue that CDC has twisted the surveillance from ADDM to obscure the autism epidemic?

Ah, the joys of deconstructing CDC statistics. How do they lie to us? Let me count the ways.

1. Start the history of autism in 1992

CDC surely understands that a huge part of the controversy over rising autism rates is defining when the increases started so we can better locate the environmental roots behind autism’s recent rise. They also understand that the criticism over CDC’s own role in this controversy begins with the escalation in the childhood immunization program that started with the introduction of new vaccines for Haemophilus influenza B (Hib) and hepatitis B in 1990 and 91, respectively. And they certainly understand that an honest investigation of the effect of their own policies must include a comparison of autism rates for children born in 1989 or earlier with those born in 1991 or later.

So what has the CDC done with ADDM? They left the investigation of children born in the 1980s out of the ADDM entirely! According to ADDM, the history of autism surveillance starts with children born in 1992. Anything before that becomes ancient history as far as CDC is concerned.

But what makes this omission even more dishonest is that the CDC began its own autism survey work with data that did include children born in the 1980s. In fact, they led one of the more thorough autism survey analyses ever conducted, one in which CDC-paid clinicians personally conducted diagnostic interviews for every suspected case of autism in a target population in Brick Township, New Jersey born between 1988 and 1995. They can even compare this NJ data with their own data from four NJ counties in ADDM (which included Ocean County and Brick Township).

And what did CDC find when they did this thorough analysis? 

• That among children born 1988 or 1989, there were exactly ZERO cases offull syndrome autism; yet by the 1993 birth year the full syndrome rate had soared to 1 in 128. 

• That cases of all autism spectrum disorders (ASDs) went from a rate of 1 in 225 among children born between 1988 and 1991 (the most precise data breakdown received from CDC) to over 1 in 80 for children born in 1992. 

• And that the rates of ASD found in its ADDM analysis of children born in NJ in 1992 and 1994, far away the highest rates in the ADDM network, may have actually understated the autism rates in some parts of NJ.

We know all this not because the CDC actually published these numbers honestly on its own (their published report on the Brick Township survey actually denied that there was an upward trend!), but because SafeMinds founder Sallie Bernard had questions about the findings, asked CDC employees for unpublished data and received some partial responses. Left to its own devices, however, the CDC chose to launch its official history of autism rates starting with children born in 1992. The only conclusion one can draw is that they don’t really want the world to know what autism rates were before 1990.

The sharp rise in NJ autism rates for children born between 1988 and 1994 is displayed in Figure 1. (Click to enlarge)

 

2. Hide behind diagnostic changes and then don’t control for their effect

The solitary argument supporting the denial of the autism crisis is the notion that autism rates have risen due to “better diagnosing” and a broadening of the diagnostic criteria for autism. The fragile factual foundation of the “better diagnosing” argument lies in the adoption of a revised set of criteria for an autism diagnosis in the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM IV). DSM IV was published in 1994 around the time the autism epidemic was first becoming visible.

There was nothing special about the 1994 revision of the DSM. First published in 1952, the DSM, the bible of the psychiatric profession, was first revised in 1967 (DSM II). “Infantile autism” was slotted within the broader category of pervasive developmental disorders (PDDs) and first formally included in the third edition, which was published in 1980. Seven years later, the revised third edition was published and with it an update of the DSM criteria for autism (renamed “autistic disorder”) and the PDDs. Then, seven years after the revised third edition came the fourth edition, with another refinement of the criteria for the PDDs and “autistic disorder.”

The key point to remember is that the DSM criteria for all “mental disorders” are revised regularly and autism is just one part of this ongoing process. If you read the literature surrounding the autism revisions, you will see that the DSM IV was never intended to radically expand the definition of autism. Quite the contrary, these revisions are mostly technical publications designed to aid practicing psychiatrists in making consistent diagnoses. But since the timing of this new volume coincided conveniently with the upsurge in autism cases, history has been rewritten. Because as the first children of the Age of Autism were born in 1991 and went in to receive their first diagnosis of autism at the average age of four or five,  the DSM IV criteria had, quite naturally, just been introduced.

Very little about autism or the PDDs changed in DSM IV. There was a bit of tightening in the checklists. The “age at onset of symptoms” threshold (implicit in Kanner’s phrase “infantile autism” but jettisoned in the revised third edition) was reintroduced and set at 36 months (DSM III had it at 30 months). The overall intention was described explicitly in 1997 by some of its designers: “The change from DSM III to DSM III-R [revised] is an example of the broadening of the concept of autism; from DSM III-R to DSM IV, a corrective narrowing occurred.” [emphasis added] 

But there was one material change in DSM IV. For the first time, Asperger’s syndrome was included as part of the PDDs. That should have been neither radical nor confusing. After all, Asperger’s syndrome, by definition, doesn’t involve the language deficits of full syndrome autism. Because of that difference (quite obvious and not confusing for parents, but seemingly the confusion at the root of the epidemic for others) the age of diagnosis usually comes much later: typically eight years of age for Asperger’s vs. three to five years of age for autism.

There’s nothing wrong with including Asperger’s syndrome in the PDDs. Indeed, formally recognizing the disability of those with only the social and behavioral deficits of the PDDs was a step forward. But that small bit of progress should never have been allowed to confuse the issue of time trends in autism. The arithmetic involved isn’t very hard.  It’s like avoiding confusion between pandemic swine flu and other upper respiratory infections like the common cold (hmm, maybe there’s a pattern HERE). If you want to control for the impact on autism surveillance data of the DSM IV inclusion of Asperger’s syndrome you can do one simple thing.

Separate diagnoses of “autistic disorder” cases from cases diagnosed with Asperger’s syndrome.

And for that matter do the same thing with PDD-NOS, which with one label or another (as “other infantile psychoses”, “infantile autism, residual state”, “atypical PDD”, or just as PDD “not otherwise specified”) has always been included in the PDDs.

But what approach did ADDM adopt? They did exactly the opposite. CDC dumped all PDD diagnoses together in one big lump, therefore specifically choosing NOT to measure the effect of the diagnostic revision they intimate has changed the rates.

Why on earth would CDC not require more precision in the gold standard of autism surveillance? They certainly know how to conduct a more detailed analysis. They did it in Brick, where they measured both the rate of all autism spectrum disorders and full syndrome autism. You can see the results in Figure 1, which shows how ASD rates in 6 year old children born in Brick Township in 1992 (1 in 82) compared with children born the same year (and surveyed two years later) in a four county sample that included Brick, (1 in 99 had ASDs). And while Brick ASD rates in the 1992 birth year are a bit higher than the ADDM rates, they show a consistent trend before (rising sharply) and after (leveling off) even though they were diagnosing children earlier (and possibly missing some milder Asperger’s cases).

The NJ data don’t give you any reason to believe the changing diagnostic criteria have anything at all to do with the rising rates. They simply show that the ADDM methods are designed only to perpetuate the confusion over what’s going on with autism.

3. If the sample doesn’t suit, change it

One of the oddities of the ADDM network is how unstable it is. The “2000” report (actually 1992 births diagnosed by 2000) lists 6 states as part of the network, a list that grew to 14 in the “2002” report (1994 births diagnosed by 2002). Given ADDM’s biennial reporting cycle, most of us who were waiting for the “2004” report last week were surprised to find the main emphasis on the “2006” numbers. This latest version included 11 states, strangely eliminating 2 of the 6 states in the 2000 group and 4 of 14 in the 2002 group (meanwhile, the numbers for “2004” were buried in an appendix; more on this below). What could possibly explain this revolving door of ADDM states? Why wouldn’t CDC simply choose a representative sample of states and counties and follow them in consistent fashion?

Well for starters, the ADDM sample is almost certainly not representative. The Department of Education (DOE) provides data on special education enrollments via the Individuals with Disabilities Education Act (IDEA) (seeFighting Autism). Based on the IDEA standard, CDC has consistently excluded from ADDM the states with the high autism prevalence rates; at no point have the top 5 states in the IDEA rankings, Oregon, Minnesota, Maine, Rhode Island and Connecticut, been included. New Jersey, by far the state with the highest rates ever surveyed by CDC barely makes the IDEA top 10. ADDM has also excluded states with their own surveillance activities. California (15th in the IDEA rankings), the state that launched the earliest alarm over rising autism rates is also not part of ADDM. Neither is Minnesota, a top 5 state that has also been the site of prior autism surveys.

In addition to excluding states with high rates and prior independent surveillance, ADDM has also manipulated the network. They have had four reporting cycles and have modified the states in each cycle, moving old states out and new ones in. There is no apparent scientific reason for these changes, nor has CDC given any explanation for the changes in the ADDM network.

So one can only assess the effect of these changes by judging the impact of their actions on the ADDM results as the network evolved over the four reporting cycles. These actions suggest that the CDC’s changes to the ADDM network have systematically reduced the ADDM reported rates and lowered their reported rate of increase.

Specifically:

• CDC has removed states with high and/or rising rates. Following the 2002 report, NJ, Utah (UT), West Virginia (WV), and Arkansas (AR) were all removed from ADDM (an action that was not visible to the public until last week). These 4 states were ranked #1, #3, #5 and #6, respectively out of the 14 states reporting results from 2002. This systematic purging of high prevalence state is visible in Figure 2 below. (Click to enlarge)

• This purge not only excluded states with high rates in 2002, it also excluded 2 of the three states with the largest rates of increase between 2000 and 2002: WV with a 39% increase in just 2 years, and NJ with a 7% increase. Only the CDC’s self-managed program in GA, with an autism growth rate of 17%, remained in ADDM.

• In 2002, CDC added Alabama (AL) to the ADDM. AL is one of the bottom 10 states in the IDEA rankings, and at the time it entered had the lowest rate in the ADDM network. CDC retained AL in the network for all subsequent reports. Most recently CDC added Florida (FL) to the ADDM in 2006. FL (which is close to the median in IDEA) entered ADDM with the lowest reported autism level of any state. 

• Pennsylvania (PA) and Colorado (CO) were both removed in 2004 (or provided no records) and then reinstated in 2006. CO is another bottom 10 state in IDEA. And although the full state of Pennsylvania is in the top 10 of the IDEA ranking, Philadelphia County (the ADDM site) has autism rates well below the statewide average. Both PA and CO autism rates were among the lowest in the latest ADDM report. (Click to enlarge)
 

In case anyone is missing the point, these are not neutral shifts. It doesn’t take a degree in statistics to figure out that if you remove the states with the highest rates in the sample, the average rates will go down; similarly, if you add in states with low autism rates, the average will fall even further.

And if you try to compare these averages while the sample is shifting, you will report misleading trends. The ADDM network estimates for 1992 birth year (2000 report) are not comparable to the latest estimates for the 1998 birth year (2006 report). Only 4 of the 6 original sites remained in 2006; and only 10 of the 14 sites reporting in 2002 remained in 2006. The exclusions and additions skewed the data in critical ways.

Even so, the ADDM rates went up anyway. Figure 2 shows that rates within each state rose consistently (every line on the chart goes up) despite the biased shifts of the overall sample. Maryland (MD), North Carolina (NC) and Missouri (MO) rates rose consistently throughout the period. Rates in South Carolina (SC) went down through 2004 and then shot up in 2006. Rates in Arizona (AZ) started out among the lowest in the ADDM, but as of 2006 are the highest in the network. Only Wisconsin (WI) shows a mixed recent trend, with a declining rate between the 2004 and 2006 reports.

In short, last week’s high autism rates would have been significantly worse had the CDC not manipulated the ADDM network to keep the rates low. As a result, some observers even called the ADDM report good news because the average rate of 1 in 110 was lower than an earlier report of 1 in 91. But that’s simply an artifact of CDC’s monkeying around with the sample: there’s little doubt that the “official” rate of 1 in 110 would have risen to over 1 in 100 with the inclusion of higher recent rates in NJ and the elimination of FL.

4. Spinning the numbers down

If states moving in and out of the ADDM network were the only evidence of manipulation, one might argue these changes were innocent coincidences, driven by a handful of unavoidable local funding problems. But the CDC’s statistical sleight of hand runs far deeper than that and is revealed most clearly in the packaging of trend information in each ADDM report release.

Trend reporting requires comparing data from different time periods. And when it comes to trend analysis, it doesn’t take a degree in statistics to know you should always compare “apples and apples” between one time period and the next. This ought to be an easy thing to accomplish in a highly organized effort like the ADDM project. But CDC has consistently twisted its reporting to suppress the evidence supporting increases in the autism rate.

Two ways this suppression has been demonstrated are revealed below in Figure 3, where the red markers and lines show what CDC reported and the black lines show what an honest, “apples to apples” comparison would have shown instead.  

1. Apples and oranges. In 2007, CDC released results simultaneously from its 2000 and 2002 reports. The initial 2000 ADDM survey contained six sites and the 2002 network was expanded to 14. When releasing the two reports on February 9, 2007, CDC’s accompanying press release noted that ASD prevalence for children born in 1992 was 6.7 per 1,000 (the “apples”), while the prevalence for children born in 1994 was 6.6 per 1,000 (the “oranges”), implying that the rate was the same (or might have even gone down) over the 2-year time period (these two numbers are represented by the two red markers in Figure 3 for 2000 and 2002). But these were not comparable numbers. A true “apples to apples” comparison of the sites included in both studies would have shown a 10% increase in those 6 sites over the two year period (the black line labeled “fixed 6 state set” in Figure 3). (Click to enlarge)
 
2. Oops, we forgot to mention 2004. But this increased rate was lower than many of us would have expected. So for nearly three years we have been waiting for the release of the 2004 numbers. It had taken six years to release the 2000 numbers and four years to release the 2002 data. But despite the natural expectation that the process might go faster, nearly five years later, the 2004 data had still not been released. To the surprise of knowledgeable observers, when the next ADDM report was issued last week, there was almost no mention at all of the 2004 findings. Instead, the entire focus was on the 2006 results. The extended delay occurred because CDC failed to issue the 2004 numbers at all! Why might they do this? One clue is revealed in Figure 3, which shows the second “apples to apples” comparison of states included in both the 2002 and 2004 surveys (the black line labeled “fixed 8 state set”). Based on an honest comparison, between 2002 and 2004, the autism rate rose by 31%. Why CDC failed to report the 2004 numbers entirely is anyone’s guess. Perhaps they hoped the numbers would flatten out in 2006. But as the red line from 2002 to 2006 shows, the upward trend simply continued and so ADDM was forced to report a 57% increase in autism last week. This time, and to their credit, the CDC’s report (the dotted red line) was a true reflection of an “apples to apples” trend. (Click to enlarge)

 

As autism rates have converged toward 1 in 100, CDC now seems to have abandoned its enthusiasm for suppressing the real autism data. Instead, it simply announced the ADDM results the week before Christmas on a Friday afternoon, hoping no one would notice they don’t have a plan to do anything about it and went back to work on the swine flu.

But it’s worth pausing for a moment to reflect on a few numbers in the latest release. Any average rate, of course, includes a mix of groups with both higher and lower risk. We all know that autism rates are higher in boys, but it’s still mind-boggling to think about the toll autism is taking in some demographic groups. The rates in the most seriously affected clusters now are over 2%. By my calculation, the autism rate among white males born in Missouri in 1998 is 1 in 45. In Arizona, it’s even higher, at 1 in 42. I know it’s not fashionable to worry these days about white male American boys. Maybe it’s time to start.

It’s night in Jungleland

The reason these male demographics stand out is because autism rates are three to four times higher, of course, in boys. They’re often (but not always) higher in white, non-Hispanic populations. They also seem to be higher in states, like NJ, with serious pollution concerns and/or aggressive vaccination policies. These two environmental factors, leading suspects in the autism epidemic, are front and center in NJ, the state with the oxymoronic nickname, “The Garden State.”

Every visitor to the Garden State who flies into Manhattan or Newark spends some time driving down the massive refinery strip on the NJ Turnpike. Bruce Springsteen, NJ’s unofficial poet laureate, wrote songs about this scar in the middle of the state’s main commercial corridor, the inescapable stigma of any New Jersey native, trying to defend their state to outside visitors. Springsteen wrote about going “New Jersey Turnpike ridin’ on a wet night ‘neath the refinery’s glow, out where the great black rivers flow.”

Not far away from the great black rivers, in the sprawl that defines the greater New York metropolitan area lies the heart of the American pharmaceutical industry. Merck’s headquarters are in Whitehouse Station, NJ. Johnson & Johnson is headquartered in New Brunswick. Major headquarters for Warner Lambert, Squibb, Wyeth and Schering-Plough were all NJ-based prior to their acquisition by larger companies and retain extensive operations all through the state. Their outsized political influence has led to unusually aggressive childhood vaccination policies.

When it comes to demographics like this, things can get personal and they do for me. I’m a white, non Hispanic male. I was born and raised in New Jersey. My father, who commuted to Manhattan his whole career, took on the chairmanship of a local autism charity when he was dying of cancer during the 1980s. One of his partners at Morgan Stanley had an autistic son, who was the only child with autism I knew who lived in my hometown. I don’t know what the autism rate was back then, no one ever did a formal survey, but it wasn’t very high. Everyone knew autism was rare.

But I was born in 1958. Just over thirty-five years later, my calculations from the ADDM report for 2004 suggest that 1 in 50 in white males born in NJ in 1994 were autistic. That was the highest rate ever reported in the US until Arizona and Missouri overtook it in last week’s report. If I had been born a few decades later, I would have fallen into that demographic. Maybe my own life would have been dramatically different. That’s enough to make you pause.

As a teenager growing up in NJ, I grew up listening to Bruce Springsteen’s music: I danced with pretty girls to Rosalita, played fake saxophone riffs along with Clarence Clemons and drank beer with my high school buddies singing along with Born to Run that “we got to get out while we’re young.” So I found myself wondering this weekend, what would The Boss think about all this? I surfed the internet reading through some of his song lyrics and found a small bit of inspiration in the lines below. I thought of changing a few words to fit the current situation, but you don’t mess with The Boss.

Outside the street’s on fire in a real death waltz
Between flesh and what’s fantasy and the poets down here
Don’t write nothing at all, they just stand back and let it all be
And in the quick of the night they reach for their moment
And try to make an honest stand but they wind up wounded, not even dead
Tonight in Jungleland

1 in 100 children. 1 in 50 boys like me. No one sounding the alarm. What is our world coming to?

Mark Blaxill is Editor-at-Large for Age of Autism and a Director of SafeMinds. To read this post in .pdf format click HERE.