An infinite regress is also an infamous phrase in the world of philosophy, where it refers to a fallacy of logic in which a premise for an argument seems to imply the neccessity of its own reapplication an infinite number of times (phew). Christian philosophers were especially concerned with this problem, as it posed a constant threat when dealing with questions regarding the existence of God...and so forth. For example, in order to avoid an infinite regress in his theory of causality (and second proof of God's existence), Thomas Aquinas proposes that there had to be an "uncaused cause" that began all change in the universe:

In the observable world causes are found ordered in a series: we never observe, nor ever could, something causing itself, for this would mean it preceded itself, and this is not possible...but if a series of causes goes on for ever it wil have no first cause, and so no intermediate causes and no last effect, which is clearly false. So we are forced to postulate some first agent cause, to which everyone gives the name God.

Others have chosen to call "it" the Big Bang, but hey. To each his own.

See also: circular logic.
Aquinas, Thomas. Selected Philosophical Writings. Timothy McDermott, Trans. Oxford: Oxford University Press, 1993, p. 200 (emphasis added).