Map of life expectancy at birth from Global Education Project.

Saturday, May 15, 2021

Epistemology V: Clinical research

The science most people probably care about the most is the endeavor to promote human health and longevity. That's why the biggest part of the federal scientific enterprise is the National Institutes of Health, with additional money going to research sponsored by the Agency for Healthcare Research and Quality, the Centers for Disease Control and Prevention, the EPA, and others.


Human health research is difficult, however. Obviously, there are stringent ethical limitations on experimenting on humans. Formally, the requirement is that there be "equipoise" -- that you sincerely don't know, or even have compelling reason to believe, which of the experimental conditions is best for the subjects. That isn't really strictly observed, in my opinion, but that's a digression. In any case it means that progress in developing and evaluating new treatments is slow, because you have to pay so much attention to safety along the way. Another difficulty is that for many kinds of treatments, and certainly for lifestyle interventions, you need the people to go home and do something, whether it be take the pills, follow a certain diet, do certain exercises, and people can't be relied upon to do that correctly. In a clinical trial, you can put in extra resources to monitor and support them, but those won't be available if the intervention makes it to the real world.


There is also what's called heterogeneity of treatment effect. A treatment will be deemed effective if the average person is better off than the placebo or alternative treatment at the end of the trial. However, some people may benefit a lot, others not at all, others may be harmed. A treatment ruled ineffective may in fact benefit some people. But you can't go back and say retrospectively that one category of people benefited after all even if the average effect was null or harmful. I'm not going to try to give you a course in statistical inference here but in short, that would violate the mathematical logic by which we decide if outcomes are real or spurious. So you'd have to do a whole new trial just to test the subgroup effect. And that's expensive.

Also expensive is long-term follow up. Typically subjects are followed for no more than six months, but obviously how well they are doing after a year or two or ten matters a lot. There is supposed to be some effort to find this out  by what's called post-marketing surveillance but for the most part the drug companies don't really bother, even when the FDA orders them to. 


There are numerous other problems. People's subjective reports of well being may be influenced by whether they think they got a treatment, and even how the experimenters interact with them. Many conditions get better on their own, so sorting out the treatment effect can be challenging, even when comparing with a placebo. And if there are adverse effects, how do you weigh the against the benefits?


Given all these difficulties, the dirty little secret is that a lot of what doctors do, even quite routinely, doesn't have strong evidence for effectiveness. And human nature being what it is, they will even keep doing things that have been shown not to work, and getting them to stop is very hard. (For one thing they get  paid for it.) So medicine is more effective today than it was 50 or 60 years ago, and it keeps getting better, but progress is slow. There's a lot wasted on ineffective treatment, and finding effective treatments is hard and slow going. But there is an increasing commitment to science and evidence based medicine. La lucha continua.

5 comments:

Don Quixote said...

After reading all that, I guess my reaction is that I wish we lived in a society where a commitment to healthful living, including focus on behavior, diet and environment, was the norm. We still seem largely to be addicted as a society to treating problems and diseases that are the results of a pathological lifestyle, instead of emphasizing prevention and true health. We seem truly myopic or even oblivious when it comes to looking elsewhere in the world for effective models to follow in any aspect of our lives, be it health care, economics, business, culture or politics. I support the concept of any program which requires one year of civil service on the part of people in high school, and as well any program of cross-cultural exchange, within the US or outside its borders. I also support the idea of any program that introduces useful concepts into the kindergarten through 12th grade curriculum, such as relationship skills, self-regulation, cooking and planting, epistemology, self-defense for girls and anyone else, and sensitivity training.

Woody Peckerwood said...

While many will agree with you, Don, in the abstract when you discuss teaching social concepts to 5 year olds, the question becomes who decides what is "useful" and what is propaganda.

My personal view is that this problem is so unsolvable that it's better to stick with the 3 R's and leave the social engineering to parents and institutions those parents might choose rather than government programs.

And I don't think I'm alone on this one.

Don Quixote said...

With the degree of alcoholism, addiction and mental illness endemic to our society, people need assistance from educators (many of whom are currently hamstrung by litigiousness and “teaching to tests”) and mental health professionals to assist them so they can recover and end the vicious cycles. Of course, I don’t speak in an abstract way. I speak as someone with firsthand experience in schools and education. We need as a society to have a much greater sense of community and interdependence in order to get help to people who need it. I’m speaking from real-world experience, not talking out my ass or theoretically. I’ve been on the front lines.

mojrim said...

That last part, estemado Cervantes, on how physicians tend to keep doing what they know how to do, reminds me of an observation I made about 10 years ago. That is, treating physicians peak around age 45. My theory is that in the years following residency they hone their skills but slowly taper off their consumption of research. The intersection of these two trends occurs after 15-20 years of practice, after which they have usually fallen off the learning curve. This is, of course, anecdotal, but it has a close fit to the experiences of myself and a number of friends.

Woody: You're certainly not alone on that but that's more a problem than anything else. It assumes that parental instruction in known behaviors have some rational basis under current conditions. The reality is that most passed-down behaviors are adaptions to adaptions to adaptions to conditions that no longer pertain. Even when they did pertain everyone was applying best guesses and heuristics with resources they had on hand. OTOH, the food pyramid (or whatever it's called now) is complete bullshit because the USDAs mission is to sell american ago product, not find you the best foods.

Cervantes said...

I don't know that it's complete bullshit, but there's a lot of room to fudge nutrition science. That's actually on deck.