Objective: To learn how Francis Galton introduced regression analysis and why the term "regression" is used.
The Giants of Potsdam
Frederick William I (1688-1740) was the second king of Prussia, and like many kings was known for his eccentric behavior. He was unusually short-tempered, possessed a host of medically illnesses, was a heavy drinker, and was an unusually cruel prankster (he once set his petite historian on fire, for a laugh).
King Frederick was also unusual in his affinity for tall men, whom he called giants. Collecting men over six feet tall (which was quite tall in the 17th Century) from all over Europe, when purchasing the giants became too expensive he would simply kidnap them. To maintain good relations with the Prussian king, the Russian Tsar (Peter the Great) would make the king an annual gift of fifty giants. These giants were toys to King Frederick, and he would dress them up in fancy uniforms, equip them with weapons, and march them around for his amusement. These toy giants were too valuable to risk placing in an actual war.
In order to maintain this legion of giants, which he called the Giants of Potsdam, Frederick attempted to breed tall men and women in hopes of acquiring tall offspring. Every tall man was ordered to marry and mate with a tall female, but the results were disappointing. After waiting fifteen years to observe the fruits of his plan, he did not produce particularly tall adolescents, and only rarely were the children as tall as their parents. In many cases the teenagers were of average height.
He started with giants, but as he bred giant with giant, their offspring regressed towards the average height of the population
Francis Galton, cousin of Charles Darwin
Charles Darwin’s cousin discovered the same phenomenon—this regression—while studying the genetics of sweet pea seeds. Francis Galton would collect data on the size of pea seeds, plant the seeds, and then study the size of their offspring. He found that peas of above-average size did produce offspring larger than the average pea, but not as large as their parents. That is, the children of large peas were a fraction smaller than the parents. Similarly, particularly small peas also produced small peas, but not as small as the parents. The children of both large and small peas regressed towards the size of the average pea.
After his experiments with peas, Galton decided to scientifically verify the story of King Frederick’s giants. After taking measurements of the heights of fathers he would then measure the height of their sons, and humans were found to regress towards the average height of the population. If the Prussian king had tried to create an army of small soldiers by forcing Prussians of low height to breed, the offspring would not resemble midgets, but would instead “regress” towards the average height of the population.
Studying the IQ of parents and their offspring Galton again found the same regression towards the mean. Two exceptionally smart parents may produce smart children, but less smart than the parents.
In all these studies, Galton measured the link between parent and offspring phenotypes through an equation, where an attribute of the offspring was stated as a mathematical equation of the parent’s offspring. It is here where the term Regression was born . Galton did not invent nor did he even use the method of minimizing the sum-of-squared prediction errors. That honor goes to the mathematician Carl Friedrich Gauss. What we call regression in this class was referred to by Gauss as the method of least squares. Today it is universally referred to a ordinary least squares, but we just use the generic term Regression.
Galton’s contribution was to measure the link between two variables through an equation. He envisioned something like Equation 1, where data on a parent and daughter seed size are linked together by a linear equation whose coefficients must be estimated from the data.
[Equation 1] Predicted Daughter Diameter = a0 + a1(Parent Diameter)
If the Prussian King had been right, and one could create tall children by breeding tall men and women (or large seeds by breeding large seeds), then Equation 1 would have the coefficient values a0 = 0 and a1 = 1. This way, the genetics of the children are mirrors of their adults. Instead, what he found was that a0 ≠ 0 and a1 < 1.
Galton never applied the method of least squares to his data. Instead, he inferred the values of a0 and a1 indirectly through data like those in Figure 1, which is an excerpt of his original study. Let us confirm the fact that a0 ≠ 0 and a1 < 1 by applying modern regression techniques to Galton’s original data (these are a subset of Galton’s entire sample). Download these data and estimate the regression in Equation 1.
Video 1—Tutorial On Estimating Regression In Excel
[Equation 1] Predicted Daughter Diameter = 12.75 + 0.27(Parent Diameter)
What this estimate shows is that even if you breed large peas to one another, over time the size of their offspring (from children, to grandchildren, to great-grandchildren, etc.) will revert to a diameter of 12.75. The coefficient 0.27 means however much the parent was distinguished from the average pea (whether it was smaller or larger than average) the daughter would only inherit 27% of this distinction.
Later we will learn that the coefficient 0.27 is really no different than zero. Consequently, the coefficient 0.27 is hardly significant, and could very well be zero. This means that if you breed two large pea seeds, there is little reason to think their offspring will be larger than average.
Figure 1—Francis Galton's 1894 Data on Pea Size Inheritance
 fig 1.jpg)
