Methodology 101, next installment: Case Control Studies

This new study is a twofer, because it illustrates the study design I wanted to write about next, and the finding is also right up our alley. Some folks in jolly old England were interested in whether pharmaceutical companies might be exercising subtle influence over medical journals by ordering thousands of reprints of studies they sponsor.

I realize this is a bit arcane to begin with. Traditional journals depend on two or three sources of revenue: subscriptions, reprints, and advertising. Subscriptions to medical journals nowadays can cost hundreds of dollars a year, which basically means that only libraries subscribe, with the exception of members of medical societies that publish journals. Only very widely read and influential journals attract significant advertising, and most of that, obviously, is from drug manufacturers. That sounds possibly bad, but in most cases it isn't obviously corrupting because it would look really bad for the companies to pull their ads if they didn't like the journal's content. (There have, however, been a couple of scandals.)

Reprints are another matter. Because these journals are very expensive, and it would be a violation of copyright to photocopy and distribute articles, if you want to get a paper into the hands of lots of people, you need to pay for reprints. Drug companies, obviously, have an interest in disseminating articles that support the use of their products, so they may purchase as many as 125,000 reprints. They aren't likely to purchase reprints of studies that aren't so favorable, obviously.

So, what is a case control study? Basically, the researchers identified papers in high impact journals for which there had been large numbers of reprint orders -- the top 20 or top 10 in a given year. In this study design, these are called the cases. Usually, "cases" are people who have a particular diagnosis. These cases are then matched with people who don't have the diagnosis, or in this instance articles for which there were not high reprint orders. You try to find controls that are as similar as possible in other characteristics that might affect the outcome. If we're talking people, you look for controls of about the same age, same body mass index if that is relevant, whatever it may be. In this instance they found the study of the same type (e.g., randomized controlled trial, editorial, etc.) that immediately preceded the case article in the same journal.

The next step is to see if there is a difference between the cases and controls in some hypothesized causal factor. This time, it's industry funding of the study. Sure enough, high reprint articles were much more likely to have been funded by the pharmaceutical industry.

A couple of points. The statistical method used in case control studies is called logistic regression, and the output is an odds ratio, not a rate ratio. Technically, you cannot calculate the rate ratio from the odds ratio unless you know the underlying rate in the total population. Let's say 2/3 of cases have the hypothesized association and 1/3 of controls have it. The rate ratio in this case is obviously 2 - twice as many cases have the thing as controls. But the odds for the cases are actually 2:1, or 2; and for controls, 1:2, or 1/2. That means the odds ratio is not 2, but 4. (Sorry to have confused you, it isn't really important.)

Second, this doesn't prove causation. There could be something else that goes along with industry funding that actually explains the result. However, editors of traditional journals obviously have a strong financial incentive to publish studies with industry sponsorship.

Open access publishing eliminates this potential source of bias. One more reason why we're all for it.