Map of life expectancy at birth from Global Education Project.

Friday, May 25, 2012


A couple of questions from commenters have inspired me to produce a bit of a primer on the kinds of evidence we use in health research. It seems to me that a lot of the political controversy over health policy and official recommendations is fueled by limited understanding of the use of evidence to make causal inferences. As I set out on this project I really have no idea how long it will take. We'll just see what happens.

Categories of evidence are commonly organized in a hierarchy, with some kinds said to be stronger or more compelling than others. I don't exactly look at it that way. Different kinds of evidence are useful in different situations and lead us to different kinds of conclusions. Just as important, how we regard any kind of evidence must depend on the prior plausibility of what it seems to show. If there is already very strong evidence for or against some conclusion, then any new evidence that contradicts what already seems highly probable is less compelling, and we should be much more diligent in looking for flaws in the study or concluding that chance alone is responsible for the observations.

This is different from confirmation bias, which is a feature of human cognition in which we tend to ignore or explain away evidence that contradicts what we already believe. We should certainly take surprising findings seriously, but we should subject them to careful scrutiny. If they do compel us to re-examine our prior beliefs, we should certainly do so. I will address these issues a bit more formally as we go along.

Alright. As most people know, conventionally anecdote is presented as the least convincing kind of evidence. An famous example of anecdotal evidence is the putative association between vaccination and autism. Parents see their child receive a shot, and some time not long after they start to see developmental regression and symptoms of autism. Unfortunately, some parents have an unshakeable conviction after this experience that the vaccine caused the autism. As I'm sure most readers already know, this is a common fallacy called post hoc ergo property hoc, because if you say it in Latin you must be really smart. I think if we started naming it in English we would still be just as smart, so I'll call it after this, therefore because of it.

While overwhelming, really incontrovertible evidence of much greater force has shown that vaccination does not cause autism, it is wrong to say that anecdotal evidence is worthless in general or should be ignored. A famous example is the efficacy of parachutes. Since we know that falling from 10,000 feet is invariably fatal, seeing a person do so with the aid of a parachute and land unharmed, even once, is extremely compelling evidence for the efficacy of parachutes. Not only has something obviously extraordinary happened, we can immediately see how it apparently works: by air resistance. The scenario makes perfect sense: it has high prior plausibility. Seeing it happen once wouldn't necessarily make me confident enough to strap on a parachute and jump out of an airplane; but it would make me believe that if the equipment were sufficiently reliable, and weather and other conditions similar to those pertaining to the observation, the parachute would be highly likely to work.

To return to the case of autism, suppose no child had ever been vaccinated and no child had ever been autistic - or at least that autism was extremely rare. If the first child ever to be vaccinated subsequently became autistic, we would properly have a very high index of suspicion that the events might be associated. If we could think of a highly plausible biological mechanism, our suspicion would be even stronger. Neither of these requirements holds in the real case, however.

Moving along, although it is often said that the plural of anecdote is not data, this is certainly false. Data is a synonym for information, and the more anecdotes, the more information. In fact the plural of anecdote can be organized formally into a kind of study called a within subjects design. I'll talk about that in the next installment.

1 comment:

thomas sabo said...

1- the pendant will be tiny, nevertheless it is known for a beautiful apple environmentally friendly colouring. The gallstones luster. The silver cheap thomas sabo polished along with extremely vivid. It can be tied in extremely strongly. Really the only purpose Now i am offering that 4starts as an alternative to a few, is that the sequence have the horrible knot inside it, that will went on about something like 20 moments along with only two units connected with tweezers to obtain out there. My spouse and i brought that bracelet intended for our besties/play sisters special birthday. Your lady loved that. What My spouse and i not noted if rehearsing alternative bracelet from the similar company will be that- persons really like the message included in the box about they really like the thomas sabo chains. Because My spouse and i reported if rehearsing one other bracelets- the bracelet are generally really lovely, the purchase price is rather good, and they occur really really really rapidly. My spouse and i unquestionably really like these types of bracelet. The photo really does that zero proper rights. In addition, Relating to the fifteen season older cousin who prefers the bracelet a great deal of, My business is ordering the girl one, while using suitable message needless to say. That bracelet is perfect for young along with older (should you consider 51 older). The message will be individual plus its just simply the all around excellent item along with you can find a great number of type that. You'll find it is a superb dimension and will in good shape virtually any wrist. It's just a win/win problem along with thomas sabo uk supplement! I got myself only two worth mentioning intended for our bff's/sisters along with My spouse and i unquestionably really like these people. It may not be about how very much the http: //www. hotthomassabosales. com/ price tag and even the grade of the bracelet After all that simply price tag $7 yet it can be additional from the that means behind offering the item. My spouse and i won't be able to hold out to offer these types of to be able to our sisters in the near future. I think they are going to really like that; )#) Excellent purchase and they arrived sooner than My spouse and i envisioned.