Map of life expectancy at birth from Global Education Project.

Tuesday, July 20, 2021

On the bias of science: Methods

This part is really complicated, and I don't think I can do it minimal justice in one post. Let me first say that there is no such thing as "the scientific method." That we continually encounter that phrase, perpetrated by people with an air of authority, is quite depressing. Again, there is no such thing. The phrase is meaningless.


Scientists use a vast array of different methods. For convenience, we can sort them into buckets, but there is diversity inside the buckets and a given study may mix and match as well. One broad division is between observational and experimental studies, but the line of demarcation isn't sharp. There are so-called quasi-experiments and purely observational findings may emerge from experimental studies. Another broad division you'll encounter is between induction and deduction. To oversimplify it a bit, induction means looking around and seeing that certain phenomena are "out there." Bees hang out around flowers all the time. If I follow them home, their hives are full of honey. That's induction. Now I can make a guess -- form a hypothesis -- maybe they use something they get from the flowers to make the honey. That's a deduction, but I need to test it. There are various ways I might go about that, which might or might not involve experimentation, actually. I'll let you think about how you might do it.


A very common kind of question, in many fields of science, is whether a group of entities that are largely similar but differ in characteristic A also tend to differ in characteristic B. We are likely to be interested in this because we want to know the cause of B. Maybe A causes B. Or maybe something else -- C -- causes both A and B. Or maybe it's just happenstance, some sort of historical accident. This sort of problem is the most interesting kind to most scientists. It is theory building and testing. 


The word theory is widely misunderstood. In the vernacular, people use it as a synonym for a hypothesis, or a guess. While scientists in everyday life may sometimes talk loosely and use the word in that way, that is not the formal meaning in science. A theory is a structure of causal relations that explains some complex phenomenon. It may be hypothetical, or it may be considered reliably confirmed. Einstein's theory of gravity, that it is caused by the warping of space-time by massive objects (with considerable additional complexity and quantification) is considered to be securely established. So is evolution. It is wrong to say these are "only a theory." They are theories that are considered, by consensus of scientists in the relevant fields, to be correct.


But again, there are many different ways of testing theories and their components, which may be experimental or observational. What matters for our purposes is that there is something of a hierarchy of prestige among investigative methods, and among categories of questions. There are also some conventions and assumptions about the appropriate ways of reporting on and interpreting observations. These tend to push scientists in certain directions which may affect the kinds of inquiries they undertake, and what they choose to believe and say about the results. I'll try to explain that next time.

3 comments:

mojrim said...

Permit me to explain what laymen like me mean when we say "scientific method" with such authority, estemado Cervantes. We are describing the overall means by which scientists arrive at laws and theories, not any specific methodology or which ones are most applicable to which fields. It describes how scientists move from observation to a testable hypothesis, not being content with just the deduction. How whatever experiment is devised to test it must be repeatable by others. How tested hypothesis relate to theories, and how those theories can never be proven but only disproven. It's in describing the difference between hypothesis, theory, and law.

Scientists don't use the term "scientific method" because it's the air they breathe, it's baked into their assumptions about how things are done. For the rest of us it provides a handy short-form when explaining, for example, that the law of gravity gives predictive power for a falling object but does not provide an explanation for the event. You cannot understand the importance of this until confronted by a relative who dismisses something as "just an unproven theory" and claims that "once proven, it becomes a law."

Cervantes said...

I believe I did explain how scientists use the word theory. A "law" is generally a component of a theory, usually expressing some quantitative relationship. Often these are idealized and reality is less precise!

There is debate about whether falsifiability is a sufficient criterion to demarcate science. You can wonk out here if you like. I personally think it is more complicated than that.

mojrim said...

While it may be more complicated in actual practice, things like falsifiability serve as good argumentative cornerstones in lay debates. That, as I have said, is the value of a term like "scientific method."

Most people, I'd argue upwards of 90%, will never understand the intricacies of this or that methodology but can be shown that it is a thing and must conform to some general outlines. From there they can be shown that whatever information they are foisting about is "unscientific" and has limited evidentiary value. While your interlocutor is unlikely to change their mind, this can sway the audience. When your goal is to persuade people to action within our political economy, as is mine, terms like this are invaluable.

Even if they don't carry strict meaning within the scientific professions.