Almost every single one of these features is an Atavism. An Atavism is the apparent reversion of an animal back to an ancient form. They normally occur when dormant genes are turned on to when a mutation in a gene causes the animal to develop the gene coding of an ancient relative.

There have been many incidents of atavisms recorded throughout history, all the way back to the time of Caesar. They have long been considered unimportant oddities. To some they were seen as a reminder that previous imperfections lay just below the surface.

Stephen J. Gould claims something else entirely. He contends that they teach an important lesson the potential result of small genetic changes. Atavisms support the belief that phenotypes can change rapidly with one or two small changes unlike the standard gradualist model of evolution where many small changes are needed over generations. Genetic systems contain extensive, hidden components and capacities for expressing small changes as large effects. That these components exist is the only way for atavisms to exist. He feels that these components, which have possibly been hidden for millions of years, would allow for rapid evolutionary changes and adaptation.

Another example of an atavism could be some Flightless Beetle’s wings. The wings are fully developed, but the cover on the backs of the beetles became fused to its shell at some point in history, rendering it flightless.

Atavism is a theory presented by, among others, Gould. It does not explain the unlikelihood of 1/3 of horses in a later German test study having the ancient gene, and the other 2/3 that had 3 toes having the 3rd toe duplicated. Also, as melknia says, the probability that the same gene mutates everytime these fairly common atavisms come up is fairly low.