Map of life expectancy at birth from Global Education Project.

Friday, June 14, 2024

Clinical Trials continued: Under Control

In standard parlance a clinical trial (and other experiments with similar structure) has at least two "arms," the intervention and the control arm. "Control" is perhaps a strange word choice in this context, and I'm not sure how it originated, but it's whatever the intervention is being compared to. If we're doing experiments with chemicals or cell cultures we could compare the intervention to doing nothing, or more precisely to keeping all conditions the same except for the intervention. 

When we're doing clinical trials, however, we're experimenting on humans, who have some properties that make the proposition of "keeping all conditions the same" complicated; and we're also inevitably doing more than just the intervention we're testing. Testing whether a pill is effective against depression or pain or high blood pressure or cancer -- whatever it may be -- means doing more than just getting the chemical into the person's body. It requires quite an elaborate interaction with the research subjects (or participants, there's debate about the right word for them). 

People who will be assigned to either arm need to go through the informed consent process, so that's the same for everybody. Ideally, assignment to the intervention or control arm happens after that. But you can't just tell the people in the control arm to go home, we'll see you in a few weeks to give you the assessment. The people in the intervention arm are going to have an interaction with a clinician, including a discussion of possible side effects and what to do if you think you're experiencing them; the importance of taking the pills on schedule; and then whatever else the person brings up and since they're talking to a doctor or a nurse that could be anything and it might require a referral. Then, ideally, you need to monitor the people in some way to make sure they really are taking the pills as prescribed. (Half of the time people don't, even in a real world context.) 

All of this attention is likely to change people's expectations, and it may even change them in other ways. If you're trying to treat depression or anxiety, it might even be effective without the pill. If you're assessment will depend even in part on self-reports, it can change those even without any biological effect. And of course, ethically, if some other problem comes up in the interaction you do have to make a referral, which means the people may be more likely to get other treatments during the course of the trial. Ergo, you need to go through the identical rigamarole with the people in the control group. Also, the people who perform the rigamarole have to be "blinded" to which condition the person is in, or they might behave differently in some subtle way. 

So that's where the whole  placebo control concept comes from. Typically, people in both arms of a trial will be found to improve over the course of the study, so we need to ask whether the improvement in the intervention arm is greater than the improvement in the control arm. But what is this "placebo effect," exactly?

There is a widespread belief that it is some mysterious "mind over body" phenomenon in which just expecting to get better heals your body. However, as far as we can tell that is rare to nonexistent, at least when we're talking about measurable biological effects. There's a bit of a philosophical problem if we're talking about effects that are only measurable by self-report, such as pain or depression. If people say they feel better then presumably they do, badabing badaboom, but if we measure, say, the actual range of motion of someone with arthritis, or the size of a tumor, or the viral load, we won't see any effect. So what is the placebo effect, really?

First of all, as I said before, most conditions get better on their own; or, in the case of chronic conditions, people are most likely to enter the trial at a time when they are experiencing an exacerbation. Either way, you're going to see improvement just with the passing of time. And often, even when we are using objective biological measures, outcome measure will also include participant self-reports so it's important to separate those out and not just put them together in an index. Finally, it's possible that the placebo affects people's behavior. If they're more optimistic about their condition, they might be more likely to take other steps to improve their health, such as eating better. They might sleep better or be more physically active. They might socialize more. 


So that's why you need a double blind placebo control. Now, there are situations in which that might be impossible or unethical. I'll talk about those next.

Thursday, June 13, 2024

Clinical Trials, Part Two: The ideal

Last time I sketched out, very briefly, some of the ways we can fool ourselves about medical interventions -- treatments. It should be a sobering thought that for 2,000 years physicians -- including people like Benjamin Rush, a Philadelphia physician who had immense prestige and signed the Declaration of Independence  -- believed in the ancient Greek Galen's humoral theory of disease, which is complete bullshit, and treated people in ways that only helped to kill them. 

The randomized controlled trial is intended to eliminate possible sources of bias or faulty inference. Although the Scottish physician James Lind famously conducted a trial of citrus fruit to treat scurvy in 1747 (and he didn't actually draw the correct conclusions from it, contrary to legend), RCTs didn't become commonplace in medical research until the 1950s, and exacting standards for their conduct were not developed until the 1990s. Now, however, in order to get published in a prestigious journal your trial has to meet the standards. In principle, the same is true for approval of a new drug or a new indication from the FDA, although in fact FDA standards can often be a bit lax. We'll get to that later.

So I'll give you a list, generated from my own head, but I believe pretty accurate, of what you need to do to meet the conventional standards of rigor. Today we'll start with:

State your hypothesis in advance: Exactly what is the indication for your treatment, including the diagnostic criteria for the condition you propose to treat, the characteristics of the population you will study (e.g. age, sex, comorbid conditions or lack thereof), reasons for exclusion, how you will try to achieve diversity within the parameters you have set. What is the outcome you are hypothesizing, e.g. cure, amelioration of symptoms, extension of life; and how exactly will these be defined and measured.

This is very important because of the rules of inference. I'm won't get into the very deep weeds of current ideas and controversies in biostatistics, but whether you're doing Gaussian or Bayesian statistics, this is absolutely essential. Since it's still the norm in RCTs to report p values and confidence intervals, without explicit consideration of prior probabilities, I'll briefly explain in qualitative terms what these mean. 

Purely by chance, the people who get the intervention -- the treatment being tested -- might turn out to do better than the people in the control group. (We'll get to the control group, including the placebo concept, later. For now we're just talking about the math.) The difference between group A, the treatment group, and group B, the control group, in the proportion who improve, or the average amount of improvement, might be caused by the intervention, or it might just be a coincidence. (It could also be that group A and group B weren't really the same in the first place, but we'll ignore that for now.)

In a random sample drawn from the population, p is the probability that you would see the difference you actually observed between Group A and Group B if there is no real difference between them in the total population. In an experiment such as an RCT, it's the probability you would see the observed difference if the treatment actually has no effect.

The smaller p is, the more likely the difference actually exists. E.g., if p=.05 (i.e. 5%), in principle (not necessarily in reality) you would see this difference only 5% of the time, if it isn't real. (That's assuming you don't have other reasons for believing it is or isn't real, which is where we get into Bayesian inference, which we aren't doing right now.)

p depends on 3 quantities:

The size of the difference between the groups. The bigger it is, the smaller p will be.

The size of the groups (n). The bigger the groups, the smaller p will be.

How “spread out” the values of B are, and of other variables in the 2 groups that might matter.

Now I'm going to give you a couple of screen shots from my lecture about this, and go away until tomorrow.