As stated above, the theory of ontogenetic recapitulation of phylogeny (hereafter referred to as ORP to make things shorter) was postulated by Ernst Haeckel. It became one of the sillier episodes of the case for biological determinism.
Biological determinism is an idea that can be traced back at least as far as Plato's Republic. It states that the economic and social differences between groups of humans (as in races, or sexes, or classes, or any other distinctions) reflect inborn traits. It has been used by reigning social groups to explain all manner of discriminatory quirks.
At the time Ernst Haeckel proposed ORP, many of the scientific minds of the age were experimenting with various ways of biologically determining that they were better than everyone else. Cesare Lombroso the anthropologist was studying criminals to find the anatomical earmarks of atavism that indicated latent criminality: tracing the bad seed, as it were. Paul Broca was using craniometry to conclude that white people had larger - and thus better - brains.
The biological determinists, armed with a priori convictions, mounds of data, and fallacious arguments, had been especially keen on ranking the human races*. Their anthropometrists, especially the craniometrists, found that the theory breathed new life into their fields of study. Everybody was now busy finding out what the Africans had or didn't that likened them to an juvenile stage of Caucasian developement. E.D. Cope, a reknowned American paleontologist wrote: "Two of the most prominent characters of the negro are those of immature stages of the Indo-European race in its characteristic types." He cites underdeveloped calves and flattened nasal bridge as examples. Continuing in the nasal vein (I am funny, n'est-ce pas?) he begins to exceed the scope of mere biology as he rhapsodizes, "The Greek nose, with its elevated bridge, coincides not only with aesthetic beauty, but with developmental perfection."
ORP as it relates to racist biological determinism was summated by the anthropologist D.G. Brinton: "The adult who retains the more numerous fetal, infantile or simian traits, is unquestionably inferior to him whose development has progressed beyond them...Measured by these criteria, the European or white race stands at the head of the list, the African or negro at its foot." However, the theory also had sexist application as America's premier psychologist of the time, G. Stanley Hall, noted: "This is one expression of a profound psychic difference between the sexes. Woman's body and soul is phyletically older and more primitive, while man is more modern, variable and less conservative."
By 1920, the theory was out of favor and the story might have ended there. But as the node above notes, this was not the end of the idea. The prevailing theory now holds that a correlation between ontogeny and phylogeny still holds true, but in the reverse relationship: phylogeny recapitulates ontogeny (PRO). This meant that species who looked more like their ancestors' juveniles were obviously 'better'**.
This contrary postulation was made by Louis Bolk, an anatomist from Holland. Bolk argued that humans were neotenous (hanging onto youth). As evidence he cited features of adult humans and juvenile apes which the apes have lost by adulthood. These include vaulted cranium, high brain:body ratio, restricted areas of hairgrowth, and the unrotated big toe. Many of the biological-deterministically minded scientists were now faced with a dilemma of comfortable-illusion crushing dimensions. They had been collecting reams of data in their efforts to prove the ascendency of the Caucasian race. In the light of this theory, the data now showed Caucasians to be the 'least advanced'.
At least one admitted the superiority of women, although avoiding making a similar admission about the Africans. The other supporters of the neoteny theory simply looked for a different way to prove the inferiority of the Africans, turning a blind eye to seventy years of research. Louis Bolk declared that neotenous races were 'better' and quickly drummed up some contradictory anthropometric data in order to show that the whites were more neotenous, claiming, "The white race appears to be the most progressive, as being the most retarded." Retarded indeed.
*Caucasian, Indian, Asian, and African were the most common groupings. They were ranked in varying orders by different scientists in varying ways, but Caucasians always came out ahead, and Africans always ended up with the short end of the stick.
**In terms of evolution, a reified 'goodness' is meaningless fallacy. Likewise, there is no such thing as 'advanced' or 'superior' species. It doesn't matter how much 'better' or more 'advanced' one species or group is than another, if that species cannot thrive in its environment then it was a bad idea. 'Good' or 'advanced' don't come into it: there is only the right tool for the right job.
The above information and quotations are all from from Stephen Jay Gould's astounding 'The Mismeasure of Man'. It is a book about the perpetuation of racism (and other discriminations) through science, but most importantly about the idea of 'objective' scientific observation amid common prior conception. It is at the same time more scholarly and interesting than this summary.