Karl Pearson and the Probability Distribution: The fact that observable events are merely random reflections

Charles Darwin recognized biological diversity as a fundamental element of life and made it the basis of his theory of the “survival of the fittest.” However, it was his fellow Englishman Karl Pearson (1857–1936) who first realized the fundamental nature of statistical models and saw how they offered something different from the deterministic scientific understanding of the 19th century. In the 1870s, as a young man, he left England to pursue graduate studies in political science and went to Germany. There, he was influenced by the works of Karl Marx and changed the spelling of his own name to Karl to show his admiration. He returned to London with a doctorate in political science and wrote two respected books in the field. In the heart of Victorian England, he had the courage to organize a Discussion Club for young men and women to gather together (unsupervised), where gender equality was inspired by the upper-class salons of German and French society. Here, young men and women debated major political and philosophical issues from around the world. I also observed this gender equality during my recent trip to Scandinavia. The fact that Pearson met his wife in this environment suggests that there may have been more than one reason for his motivation to establish this club. This small social initiative can be an example for understanding Karl Pearson’s original mind and his absolute indifference to established traditions.

Although his doctorate was in political science, Pearson’s main interests were the philosophy of science and the nature of mathematical modeling. In the 1880s, he published his book “The Grammar of Science,” which went through several editions. In the period before World War I, this book was considered one of the great works written on the nature of science and mathematics. The book was full of brilliant and original insights and was seen as an important study in the field of the philosophy of science. It was written in a style simple and fluid enough for everyone to read and understand. You do not need to know mathematics to read and understand “The Grammar of Science.” Although the book has existed for more than a hundred years, the insights and ideas within it are still valid for most of the mathematical research of the 21st century and have maintained their accuracy to this day in understanding the nature of science.

With the correlation formula I mentioned in my article last week, Galton had come very close to a revolutionary new idea that would change almost all the sciences of the 20th century. However, it was his student, Karl Pearson, who first formulated this idea in its most complete form. To understand this revolutionary idea, we need to set aside all our prejudices about science. As we are often taught, science is about measurement. 

We make careful measurements to find mathematical formulas that describe nature. In high school, we are taught that the distance a falling object travels over time is calculated with a formula containing a symbol called g. This g is the constant of acceleration. We are taught that the value of g can be determined using experiments. But what happens when a high school student performs a series of experiments to determine the value of g, rolling small weights down an inclined plane and measuring how long it takes them to reach different parts of the ramp? The results usually do not come out correct. The more the student repeats the experiment, the more confused they become as the resulting g values turn out differently in different experiments. The teacher says that the reason the students could not reach the correct result is carelessness, sloppiness, or copying the wrong numbers. 

However, what the teacher does not tell the students is that all experiments can be flawed and that even the most careful scientist rarely obtains the exact number. In every experiment, small, unpredictable, and unobservable errors occur. The air in the room might be too warm, and the sliding weight might pause for a microsecond before it starts sliding. Even the slight breeze of a passing butterfly can have an effect. In reality, what is obtained from an experiment is a distribution of numbers that are not exact. However, these numbers can be used to obtain a close estimate of the true value. This explanation is very important. 

According to Pearson’s revolutionary thinking, we do not look at experimental results as carefully measured numbers on their own. Instead, we say these are a sample of a distribution of numbers; to use an accepted term, these results are samples of a distribution. This distribution of numbers can be written as a mathematical formula that tells the probability of an observed number having a certain value. “It cannot be predicted in advance what value a number will take in a particular experiment. We can only talk about the probabilities of the values, not their certainties. The results of individual experiments are random; in this sense, they are unpredictable. However, statistical models of distributions allow us to define the mathematical nature of this randomness.”

Pearson thought that measurements themselves, rather than measurement errors, have a probability distribution. Whatever we measure, it is actually part of a distribution of randomness, and these probabilities are defined by a distribution function, which is a mathematical function. Pearson discovered a series of distribution functions that he called “skewed distributions,” which he claimed would describe any kind of dispersion a scientist might see in data. Each distribution in this family is defined by four numbers. The numbers that define the distribution function are not “numbers” of the measurement type. These numbers are unobservable but can be inferred from how the measurements are distributed. These numbers were later called “parameters,” of Greek origin, meaning “almost measurement.” The four parameters that completely define a distribution of the Pearson System are:

1. Mean - the central value around which the measurements are distributed,

2. Standard deviation - how much the measurements are spread around the mean,

3. Symmetry - how much the measurements are piled up on only one side of the mean,

4. Kurtosis - shows how far rare measurements deviate from the mean.

A subtle shift occurred in thinking with Pearson’s system of skewed distributions. Before Pearson, the “things” that science dealt with were real and concrete. Kepler tried to discover the mathematical laws that describe how planets move in space. William Harvey’s experiments tried to determine how blood moves in the veins and arteries of an animal. Chemistry was concerned with elements and compounds made of elements. However, the “planets” that Kepler was trying to understand were actually a series of numbers that determined the positions in the sky of the flickering lights that observers saw on Earth. The exact path of blood flowing through the veins of a single horse was different from what might be seen in a different horse or a human. No one could produce a sample of pure iron, but it was known that it was an element.

Pearson suggested that these observable events were merely random reflections. The thing that was real was the probability distribution. The real “things” of science are not the things we can observe and hold, but the mathematical functions that describe the randomness of what we can observe. 

Therefore, the four parameters of a distribution are what we really want to determine in a scientific study. In fact, we cannot truly determine these four parameters. We can only estimate them from the data. Pearson failed to realize this final distinction. He believed that if we collected enough data, the estimates of the parameters would give us the true values of the parameters. However, we can only approximate. We cannot know for sure. His young rival, Ronald Aylmer Fisher, emerged to show that many of Pearson’s estimation methods were not optimal. I will talk about him in another article. In the late 1930s, as Karl Pearson approached the end of his long life, a young Polish mathematician, Jerzy Neyman, also showed that Pearson’s system of skewed distributions did not cover the universe of possible distributions and that many important problems could not be solved using the Pearson system. This is the process of scientific progress. It always advances by building upon existing knowledge.