More on valuing health outcomes

Okay, so now for some of the low-hanging fruit.*

Ideally -- but not necessarily really -- the way treatments such as new drugs are evaluated is through randomized controlled trials (RCTs), conducted according to certain standards. One of the most important of these (and probably most often violated, in the past at least) is that the trial must test pre-specified outcomes. The reason for this stringent requirement has to do with the rules of inference. Whoo. Here goes some headbanging.

As I presume most readers basically know, the way you do an RCT is to divide a bunch of people at random into two groups. One of them gets the magic potion, and the other doesn't. Whatever your specified outcome may be, there's a chance that even if the stuff doesn't work, more people in the intervention arm will have the outcome, just by random variation. You can calculate what that probability is, given the observed difference between the two groups. It's called the p value. Pretty much arbitrarily, we say it has to be less than 5% (p < .05) in order to call the trial successful.

Here's where the big problem comes in. There are bound to be some differences, actually a lot of differences, between the two groups. If you go looking around for them until you find one, and then calculate its p value as if you had specified the outcome in advance, that p value is bo - o - o - gus. Basically, it's meaningless.

Here's another problem. In order to get a drug approved, companies have to show that based on the above procedure, it's better than placebo. But they don't have to show that it's better than an existing, quite likely cheaper, alternative. They also don't have to show that the benefit is of any particular magnitude, and they don't even necessarily have to show that it actually benefits people at all. They can rely on so-called "surrogate end points," that is, indicators that are thought to be predictive of better health outcomes, such as lower LDL ("bad" cholesterol), but which might not actually be better after all. More than once, a drug approved on the basis of a surrogate endpoint has ultimately been found not to produce the expected better outcome.

So . .. Long story short, there are quite a few interventions out there that really aren't better than cheaper ones, or which don't really do any good at all for most of the people who get them. There is a voluntary effort now by many of the medical societies to encourage their members not to use some of these, but no general authority that says, for example, that Medicare won't pay for them.

That's because Congress forbids Medicare from taking cost into account in deciding whether it will pay for treatments. We could start by not paying for stuff that isn't any better than cheaper stuff, or that really doesn't do any good. But as soon as somebody proposes it, we get ignorant idiots screaming about death panels.

Next I'll talk about some harder issues, since this one ought to be easy.





* George Orwell advised "never use a metaphor, simile or other figure of speech which you are used to seeing in print," in the interest of avoiding cliched writing. Sorry George, just couldn't find a better alternative.