Spin Doctors

This page has from time to time opined about the pervasive overhyping of medical research by the corporate media. Every experiment on mice or Phase II trial gets reported as promising to eliminate cancer or make the blind see.

I have mostly assigned the blame to journalistic credulity or the commercial value of sensationalism, but this study in PLoS Medicine puts the blame elsewhere -- in the actual abstracts of the published studies. In my view, that doesn't let the reporters off the hook -- if they're going to write about medical science, they should be competent enough to read the whole paper, understand it critically, and interview the authors with perspicacity. (Look it up when you get home.)

But, there is something to this. I cannot tell you how many times I've downloaded a bunch of abstracts, gotten the articles that looked relevant for whatever I was thinking about, and found out that half of them don't deliver what the abstract promises. This is a big problem for several reasons, not the least of which is that few practicing physicians have time (or for that matter, the technical knowledge of research methods and statistics) to read research reports. They just read the abstracts. The presence of research results in the world is 99% just the abstract; hardly anybody actually reads the papers.

This happens because peer reviewers and journal editors, in my experience at least, pay no attention to the abstract. Journals usually have a required outline -- something like background, methods, results, and discussion -- but they have no standards about how the abstract should reflect what's in the paper. Abstracts often announce results on endpoints that weren't specified in advance, with p values. Without bothering your beautiful mind once again with Bayes, this is totally inappropriate. They basically never discuss limitations, alternate interpretations, adverse effects, or prior plausibility. They exist to sell the study, not to summarize it.

It seems to me there need to be standards about this in medical publishing. It's long overdue.