Clinical Trials Part Three: Assessment

As I have said, to do experiments rigorously you need to specify in advance the outcomes you will be measuring, and how you will measure them. (A fancy word for that is operationalizing your variables.) For example, if you want to know if your magic pill is effective at treating Covid-19, you need to be precise about the outcomes you will be comparing between the intervention and comparison groups. You also need to be precise about the starting condition of the population you will be studying. Are we talking about people with mild to moderate disease, who would not normally be hospitalized? People who are already in the hospital? How do you define and measure disease severity?

Let's say we're going with an outpatient population. You need to test them so you know they actually all have Covid-19, and you need some reasonably objective measure of disease severity. Oxygen saturation, viral load, a structured questionnaire assessing symptoms, that sort of thing. You need to specify how long you will follow the people, and how you will define your outcomes. For example, proportion who end up being hospitalized, days to recovery as defined by a pre-specified response to your questionnaire. Oh yeah, this is very important: the person who does the assessment, such as administering the questionnaire, must not be an investigator and must be blinded to the trial arm. The reason is that someone who knows who is in the intervention vs. control group, and has a stake in the outcome, can easily skew the result, even if unconsciously. 

Usually, preliminary studies are put together quickly, have little or no funding, and so have small sample sizes and less than rigorous design. That is why typically early, quick and dirty studies report positive results that don't hold up when people try to replicate them more rigorously. And that is what happened with ivermectin and hydroxychloroquine. Some ambitious docs did poorly conducted trials and announced that the drugs were effective. Actually, some of these weren't even controlled at all, it was just "I give this to my patients and they recover." Well yeah, most people do. 

 

And that is why somebody thought they had a gotcha when they sent a link to a study that found hydroxychloroquine to be effective against Covid-19. No, it isn't. And the people who made that claim ended up wasting millions of dollars and urgently needed investigator time and resources because NIH felt compelled to fund rigorous studies of what were essentially folk remedies, that as it turns out absolutely, incontrovertibly, do not work. It is also highly irresponsible for elected officials to promote unproven treatments. 

Now that I've given an idea of what constitutes a well designed and well conducted trial, I'll go on to the limitations of this kind of study, what it doesn't tell us, and when it's actually impossible. What else can we do?