Deep Thought

No, the title does not signal an inane quip, I really do want to start my discussion of probability and statistics with some musings on the philosophy of science. I'll try to keep it all as clear and bullshit free as possible. Let me know if I fail.

For a long time, as people did what we today recognize as science or something like it, philosophers didn't fire up any neurons worrying about what they were doing. Archimedes, Eratosthenes, Galileo, Newton, Herschel, Darwin, pretty much just did their thing, while philosophers worried about the moral virtues and the noumenon and what not. By the 20th Century, however, distinguishing between scientific and unscientific beliefs and methods of inquiry had obviously gotten to be important. Philosophers came up with an idea called positivism, which in a nutshell held that a statement is meaningful only if it can be verified, and that it's meaning is equivalent to the means by which it can be verified.

This would seem to imply that there is some stuff that we can totally know -- assertions that can be fully established as true -- and that any other kind of assertion is just gobbledygook of no interest to those who wish to live by the light of reason. However, it soon became clear that this doesn't really work. The kinds of general statements about the world that constitute interesting science are difficult or maybe even impossible to prove in a formal sense. Newton's theory of gravitation, as a matter of fact, was not exactly true. It was true as far as anyone could tell with the accuracy of measurement and the conditions available to Newton's scrutiny but then along came Einstein and with better instruments we found that his theory is more accurate.

But does that mean that Einstein's theory is proved, or that even in principle we will ever have the means to demonstrate as a matter of formal certainty that it isn't wrong somehow, somewhere? Not at all. In fact physicists have been working very hard for a long time to do Einstein one better, and develop a theory that unifies gravity with the other cosmic forces. So they certainly don't think Einstein's theory is a settled matter. But does that mean it is not a scientific theory? Obviously not.

So Karl Popper proposed that at the very least, even if we couldn't really prove anything, a scientific proposition had to be falsifiable. There had to be tests available to show that it was not so. But that turned out to be a bit of an oversimplification as well. One spanner was thrown into the works by the Austrian mathematician Kurt Goedel, who proved that in any logical system at least as complicated as number theory -- basically everything you can say about numbers using arithmetic -- there are true propositions that can never be proved, which implies that their negations -- false propositions -- can never be falsified. To be sure, he's talking about abstract systems of deduction, not empirical investigation per se, but scientists use mathematical reasoning all the time to describe the world, plugging empirical entities into formal systems of logic. So Goedel's theorem (called the Incompleteness of First Order Arithmetic) was a bit disturbing.

Another problem was that physicists discovered that the world at very small scales -- where the electrons and protons and photons disport themselves -- is not deterministic, but can only be described in terms of probabilities. By the time you pile up enough atoms to make an object apprehensible to the senses, those probabilities average out so that it is extremely unlikely your coffee cup will suddenly jump off the table. But it is only extremely unlikely, it is not officially impossible.

Since the very tiny quantum world and the larger scale world are linked, this means that the universe as a whole does not have the deterministic quality that animated positivism and related philosophical schools. People used to think that if you could know the position and momentum of every particle in the universe, and you had a sufficiently powerful computer, you could know the future. But it turns out that you can't simultaneously know the position and momentum of even a single particle, nor can you know when it might decay or change its energy state. You can only make a probabilistic statement about such events. If you wanted to assure the unpredictability of the future, you could rig up a detector that would respond to a radioactive decay event by tripping a switch that would do something really, really big, like blow up a dam; and you could have it wait for an event with a 50% probability of happening at any time within 100 years. Hmmm.

Absent such exotic experiments, the macro world is not so uncertain. If you knew all of the forces acting on a roulette wheel and the ball -- the precise coefficient of friction of the bearings, if you could map out the turbulence of the air around it and precisely how it affects the motion of the ball and the slowing of the wheel, the coefficient of restitution of the ball vis a vis whatever parts of the wheel it might touch, and so on, and the exact vector of the ball and speed of the wheel at the beginning of the spin, in principle you could predict what number will come up. But you can't possibly know all that. The tiniest variation in initial conditions, below what you can possibly measure, will produce a completely different result. Like the weather, it is a so-called chaotic system.

What does it mean when the guy on TV with the perfect teeth and the hairpiece molded from a single piece of plastic says there is a 50% chance of rain tomorrow? How can we verify his statement? Either it rains tomorrow or it doesn't; either way, he was right. So is the statement meaningless? No. What it means is that if he makes that prediction 100 times, we should expect it to rain about 50 times. It might not rain exactly 50 times -- maybe it rains 52 times, or 47 times, and we'll let him get away with it. But if it rains 80 times, we'll decide he's a lousy forecaster.

Similarly, assuming there are 38 slots on the roulette wheel, we'll call it honest if, in a thousand spins, each number comes up about 1/38 of the time. Unfortunately, 1,000 is not divisible by 38, so each number cannot possibly come up the same number of times. We'd like it if most hit 26 times and some hit 27, but that's actually very unlikely to happen. You'll most likely have some 29s or 30s, and some 23s and 24s in there, and maybe even bigger deviations. That's why once in a while, somebody walks away from the table a winner.

It turns out that science depends very heavily on probabilities. When we observe variations in populations -- whether they be of people or of stars -- we can only estimate the probability that the same patterns pertain to the people or stars we haven't observed. We can't really prove anything, or necessarily know how to falsify most of the interesting scientific theories we might come up with. That doesn't mean we might not end up falsifying any of them eventually, it just means we don't have to know how to do it when we first assert them in order for the assertion to be meaningful.

That's more than enough for now.