Inference in biomedicine

Okay, we're making progress. The next step in a standard treatment development process is called a Phase Two trial. These normally have about 100 to 300 participants, and yes, they're normally randomized controlled trials. Everybody in a Phase Two trial is diagnosed with the disease, so we're looking for preliminary evidence of efficacy. Half (normally) of the people will actually get the treatment, and half will get either a fake treatment (i.e. a placebo) or a standard treatment against which the new drug is being compared. The latter will often be necessary for ethical reasons if an effective treatment already exists, certainly so if the condition is dangerous if untreated.

Now here we run into a whole lot of problems that chemists aren't trained to think about. The first is establishing the eligibility standards for the trial. Every atom of iodine and every molecule of hyaluronic acid is identical, but every person with Creeping Crud is different. Different ages, different sexes, different ages, height, weight, other diseases and overall state of health, different health-related habits, different severity of Creeping Crud. In fact, we might not be 100% sure that all of them actually have Creeping Crud since diagnosis is not 100% reliable. Obviously, these could all affect how people respond to the treatment. 

This is a mathematical problem. The more variables to consider in addition to the intervention you're trying to evaluat, e, the less your power to establish the effect. For this reason, investigators might want to limit the participants to, say, people with greater severity of disease, good health otherwise, under age 55, and so on. Once you've done that, you can divide the people at random into your treatment and control arms. This doesn't guarantee by any means that the various potentially "confounding" factors will be evenly distributed between the two groups, but at least it's better than intentionally assigning them in some way that could skew the results.

But you still have more problems. For most diseases, people have good days and bad days. There are periods where the symptoms are worse and periods where they remit. Some conditions, of course, can be entirely time limited and just get better. If you've limited your sample to people with a certain level of severity, some of them are going to be better off the next time you measure them even without treatment. That's one reason for the placebo control. But even so, it could just be a coincidence that more people in the treatment group improve. (And it could go the other way -- maybe the placebo will end up looking more effective. That does happen.) 

Another reason for placebo control is that measures of efficacy often depend, in part or in whole, on participant self-reports. How much pain are they feeling, how much energy do they have, are they depressed? We know that if people think they're doing something that might be effective, they will report feeling better. If they're in a clinical trial, they're also getting a lot of extra attention from doctors and nurses, which not only might make them feel better but also means that whatever other health problems that may arise will get a quick response, which might make them healthier. (Or not, but that's another story.) 

Again, for any of these effects, there's at least a 50/50 chance it will be more powerful in the treatment arm than in the control arm. Also, if the people who are measuring the effect know who is in the treatment and who is in the control arm, they might be unconsciously biased to favor the treatment. (This has been proven to be true.) So the practice is to blind both participants and assessors to who is in which group ("double blind"), but this often fails in practice -- there are ways to tell. I'm not going to get into a deep philosophical discussion of the concept of statistical significance, at least not today, but the bottom line is that in a trial with 300 or fewer participants, the chance that you will get really compelling evidence of effectiveness is small. That's okay -- you're not looking to get FDA approval from this trial, that requires another step, which I'll get to. (Actually only about 1/3 of treatments go on to Phase 3 trials.)

But if you're wanting to sell a "supplement" that doesn't need FDA approval, like Linus Pauling, you're never going to do a Phase Three style trial, which typically has thousands of participants, can take several years, and costs millions of dollars. And the trials you actually do and publicize aren't going to have nearly the rigor of the Phase Two trials done by drug companies seeking FDA approval. What's more, you can do more than one. As I say, it's very easy for a trial to come out positive by coincidence, so you just publicize the ones that make your snake oil look good and bury the ones that don't. (Drug companies used to do this as well, but now they have to register their trials in advance so they can't.)

I'll have a good deal more to say about clinical trials, pharmaceuticals and snake oil, but I hope this is sufficient background for that discussion to continue.