Cross of Gold -- Cont.

So okay, so what's up with the Gold Standard?

By assigning people to the intervention and control groups at random, we don't assure that the two groups are the same, except for the intervention. In fact, we know they aren't. Just by chance, they will differ in innumerable ways. Biological systems such as human beings are inconceivably complicated, far too complicated for us to be able to describe more than a small fraction of them, so we won't even be able to measure, or for that matter, even imagine all the ways in which people differ that might affect how they respond to the drug, and all of those characteristics will just happen to be distributed a little bit differently between the intervention and control groups. What is more, the people's histories -- what happens to them, what they eat, what other diseases they get, whether they get hit by a bus, you name it -- will also be different through the trial, and these histories will just happen to be different in ways that matter between the two groups.

So what do we really gain by random assignment? We gain the ability to use certain mathematical tools that are well understood, based on probability theory, to assess the probability that the differences we observe at the end of the trial between the two groups are due to the drug, or just due to chance. And by chance, we don't actually mean God playing dice with the universe -- we really mean factors that we haven't measured, or at least haven't counted in our calculation. If the probability is below a certain arbitrary standard, we say that the difference between the two groups is statistically significant.

Statistically significant doesn't necessarily mean large. If the size of the two groups is large enough, a small difference between them can be statistically significant. Also, it doesn't mean that everybody in the intervention group benefited (or was harmed, if that's the way it turned out). It could be that a few benefited, and most got no benefit. It could even mean that some benefited, and some were harmed, but the average difference was in the desired direction.

As it turns out, this is apparently what happened in the trials of a relatively new class of anti-depressants called Selective Serotonin Reuptake Inhibitors (SSRIs). There is a very large placebo response to anti-depressants -- give a depressed person a pill, and tell them it might make them feel better, and around half the people will report feeling better. But a few more people in the intervention group said they felt better than did people in the control group. The estimates based on analysis of multiple trials are that about 15% of moderately depressed people get some benefit from taking SSRIs. That obviously goes far beyond the massive hype that accompanied the appearance of these drugs, with some people saying we ought to put them in the water supply to make the whole world happier.

Some later trials, it turns out, seemed to show no benefit at all -- placebo was actually better, in one study! And then it emerged that, at least for young people, a few were actually harmed. Giving SSRIs to depressed people may make a few of them feel better, but it will also make a few -- evidently a smaller number, but more than zero -- feel worse, even worse enough to try to kill themselves. This is a short-term effect -- it is possible that in the long run, these drugs actually reduce the risk of suicide, but we don't really know yet.

So how is it that we heard one story when the drugs were first approved and marketed, and we're hearing a somewhat different story now?