ClinicalTrials.gov (wonky)

Krugman does the "wonky" trigger warning on his blog so I figured I should too. A major problem with biomedical research is what's called "publication bias." This takes a couple of forms. One is that negative findings -- i.e. "A does not cause B" or "is not associated with B" -- are unlikely to be published. Journal editors and reviewers just don't think they're exciting. However, if there are three trials that show that A does not cause B and one that does, and the last one is the only one that gets published, we wind up with a false view of reality.

A second form is that -- oh, did I ever mention that drug companies are evil? They choose not to publish studies that are unfavorable to their products. Same result, we come to believe that drugs are much more effective than they really are.

Another problem is post hoc analysis. If your initial hypothesis isn't borne out, dredge through the data to find some sort of significant association, say with a sub-group or a variable that you originally intended as a covariate rather than an outcome. The problem with this is that the p values are spurious, because if you make a large number of comparisons some associations will appear significant just by chance, when nothing is really going on.

Recognizing these problems, the FDA requires that all clinical trials for drugs be registered in advance, so that a) we'll know what the original hypotheses and protocols were and b)  we'll know about trials that aren't published. The penalty for not reporting your results within a year is supposed to be $10,000 a day.

Surprise! It isn't happening, and the law is not being enforced. We know that results aren't being reported in many cases, but we don't know how many trials aren't even being registered in the first place. Things appear to be getting a bit better, but there are still a lot of drugs being prescribed that probably should not be. This is a huge scandal that is almost completely ignored while we obsess over Hillary's e-mails.