Back on Track: Cross of Gold

Randomized controlled trials (RCTs) are said to be the "gold standard" of evidence.  (Which is a misapplication of the term, BTW, a gold standard means a monetary system in which the unit of account is based on a quantity of gold. Doesn't really make sense. But I digress.) However, if we were dependent on them for our biomedical knowledge we wouldn't actually know very much. In fact, that we depend on them so much constrains our knowledge. Let me first give a bullet list of problems and limitations. Then I'll unpack it.

1. They're very expensive. This is a really big problem because pharmaceutical companies won't do them unless there's a bit pot of gold at the end of the rainbow. That means, for example, that they are much less interested in drugs that cure quickly than they are in drugs you have to take for the rest of your life. And lots of other reasons why the cheapest and best drugs don't even make it to market.
2. The strictly defined and controlled conditions of RCTs are not the conditions in the real world in which interventions will actually be used. What works in an RCT might not work as well, or at all, in the wild.
3. To provide for cleaner inference, the people who are eligible for RCTs are usually not typical. For example, people with certain comorbidities may be excluded, or women of childbearing age, or children. Also, for various reasons, some categories of people may be underrepresented.
4. RCT results are confounded by "heterogeneity of treatment effect." Some people may respond well to a treatment, others not at all, or they have severe side effects, or the original problem can even get worse. When this averages out or dilutes the effect, we may not be able to see that the treatment really does work for some people. Post-hoc analyses are statistically invalid, so if it looks as though a subgroup may benefit, you have to go back and do a new trial just to confirm that. Again, that's expensive, and if the number of people who may benefit is small, the drug companies won't invest the money.
5. In many situations, it's literally impossible to do a RCT. It may be impossible to blind subjects or assessors, it may be unethical to have an adequate comparison,  it may not be possible to assign people at random, and there are other problems.

I'll start with the basic question of money in the next post.