Rummaging through the cortex

I covered this essay by John Ioannidis when it first came out, but I've been thinking about it recently. He claims that most published research findings are false, which may be an exaggeration, but the argument is nevertheless important. You will probably find it much easier to understand if I tell you -- as he does not, unfortunately -- that it is largely based on Bayes' theorem. If you read my little introduction to Bayes first, and then read the article again, it will suddenly be intelligible. Promise.

Anyway, the basic point is that most of the hypotheses we can make up are false, because the universe of our imagining is close to infinite. Put it another way, most syntactically well-formed assertions are false -- indeed, the percentage that are true is minute. The conventional p value to accept a finding as valid is .05, in other words the specificity of our tests need be no better than .95. Bayes tells us that a test with specificity of .95 and a prior probability of 1 in a million is is going to give a wrong result about 50,000 times for every time it is correct.

Fortunately, a lot of scientific research is not like that -- we do invest in testing hypotheses that we already have good reason to believe are true. Warning to post-modernists, creationists, and anti-vaccine wackos: There is a helluva lot that we really, truly, do know.

But there's a lot of exploratory research that gets published, and even publicized, as if its findings are definitive. Unfortunately, when it comes to health advice, that isn't good enough. It's a combination of scientists rushing to claim credit and reporters rushing to hype exciting news that gets us into trouble.

In other news, I'm really, really starting to worry. Not that I wasn't already. But the economic news is terrible -- much more terribler than the corporate media is letting on. How the public is going to react when, after all the stimulus spending and bailout and re-regulation and Easter Bunny and Santa Claus, they're still out of work and the tent cities are growing, I don't know but the speculation is killing me.

That's the Republican game plan -- they know their psychotic spending freeze and tax cuts for the rich aren't going to happen, but when Obama's policies fall short, they'll claim their ideas would have worked, even though in fact they would have made things even worse than they are going to be. And Chris Matthews will go along with it.