Cross of Gold

Sorry for the wonky post which follows, but sometimes it's necessary. Some readers will know all about this; please bear with me or just skip it.

The so-called "Gold Standard" for determining the effectiveness of any medical intervention, including drugs, is the Randomized Controlled Trial -- the RCT. RCTs are the only way to get FDA approval.

To evaluate new drugs, researchers first feed them to a small number of people to see if anything obviously horrible happens, and to get some basic data on how they are metabolized, etc. If nobody has any parts fall off, they go to a Phase II trial, which is a relatively small-scale rehearsal for the big event in which they are looking for some evidence that the stuff actually works in some way -- not even necessarily by curing or preventing disease, possibly just by having some predicted biological effect thought to be beneficial in the longer term. And of course they're still presumably alert to any adverse effects, this time looking at more people for a little bit longer.

Then they go to the Big Show. You have to do the following:

* Specify, in advance, the results that you are predicting and wish to test.

* Recruit a bunch of people who meet some eligibility standards. That may include having a disease, not having certain other diseases or conditions, being within a certain age range, not taking certain other drugs, not having a problem such as addiction or mental illness which might interfere with "adherence to the protocol," etc.

* Randomly assign them to receive the new drug, or to some comparison treatment, which could be an approved treatment, or a fake treatment (placebo). It is essential that the assignment be purely by chance, and that neither the subject, nor the physicians and other researchers who work with the subject, know what the person is taking. Also, at this point, collect a bunch of information about the people, such as sex, height and weight, ethnicity, medical history -- whatever you think might matter somehow. Of course, you're only guessing about that.

* Over the course of the trial, keep careful track of the people: if they drop out, if blood tests show they aren't really taking the stuff right, if they have symptoms that might be side effects, and of course, at the end, if whatever you wanted to accomplish by giving them the drug has actually happened or not.

* Finally, open the secret envelope and find out who was taking the drug and who was in the control group. Do statistical analyses to see if there's a difference between the groups, specifically the kind of difference you predicted, such as they're more likely to have grown hair on their bald heads or to have been cured of cancer. Also, compare the reported negative symptoms of the intervention and control groups to see if there are adverse side effects that can be attributed to the drug.

Now, this sounds like a pretty good way of doing a test, and there are indeed strong arguments in favor of it. But there are also lots of problems with it, both in principle, and even more so in the real world. I'll wait a bit to see if anyone cares to comment, before I do.