A report today in the New England Journal of Medicine, free to the uncredentialed masses because the editors have been forced to yield to my relentless pressure (credit where it's due, masters of NEJM, but you still need to make the whole journal open access, all the time), is guaranteed to achieve the exact opposite of enlightenment and clarification.
This is a perfectly well done study and the results, to people who understand statistics, are fairly clear. The problem is, that's very few people. This is what we call a retrospective cohort study, in which they examined a lot of children age 7-10 on various measures of neurological functioning, and also reconstructed their history of exposure to thimerosal in vaccines -- that mercury-based perservative which a whole lot of distraught parent blame for their children's autism. (This study did not include autism as an end point because a separate CDC-funded study of that specific issue is underway.)
On most measures, there were no statistically significant differences associated with thimerosal exposure, but on a few there were small but statistically significant differences, about equally divided between apparent benefits and apparent deficits. In other words, if you believe these results, thimerosal is actually good for kids in some respects, and bad for them in others.
But, as the authors explain in passing at the very end, and as I knew long before I got there, you shouldn't believe the results. "Statistical signficance" refers to an arbitrary threshold of probability that an observed result is not due to chance. When most people hear that a difference is "significant," they interpret that to mean that it matters, that it is substantial. But that is not necessarily true. A difference that is too small to matter clinically could be statistically significant if you have a fairly large sample, as in this case.
What is even more important in this case, if you make numerous comparisons, some differences that are in fact due to chance will appear to be "significant." If you try something that has a fairly low probability a sufficient number of times, it's eventually going to happen. Some of your scratch tickets will pay off; the Buffalo Bills might win a football game. In this study, the authors made something like 400 different comparisons, and 19 of the differences turned out to be significant -- exactly what you would expect if thimerosal in fact had no effect whatsoever.
And really, is it plausible that thimerosal would be beneficial in some respects and harmful in others? And that the effects would only show up in one sex or the other, but never both? Pish tosh.
But sure enough, after I had made my prediction, I happened to read the New York Times article on the study.
But Sallie Bernard, executive director of SafeMinds, a nonprofit parent organization whose members contend that thimerosal injured their children, said the study was inconclusive. Ms. Bernard served on a board of consultants that helped design and oversee the study, but she withdrew her support for the published version of the study, saying its conclusions were not supported by the underlying data.
“There are some red flags here,” Ms. Bernard said.
Nearly 5,000 families have filed claims with the federal government contending that vaccines caused their children to become autistic. Even if the government dismisses their claims, many families have vowed to continue their fight in the courts.
What we have here is yet another battle between faith and science. Sallie Bernard believes. She will always believe. Faith is unassailable.
No comments:
Post a Comment