Naturally, Terry Pratchett's Discworld universe has made use of this saying. Great A'tuin, of course, doesn't need to stand on anything, seeing as he (or possibly she) is swimming through space, and not standing on anything at all.
In The Science of Discworld the co-authors (Ian Stewart and Jack Cohen, doncha know - terrific scientific writers) are discussing the origin-of-time dismissal answer of saying, "it's always been there, it doesn't need explaining!" Here's how the refutation works:
Take the phrase, "It's turtles all the way down." Sounds silly, doesn't it? Not something a right-minded person like you would think, oh argument dismisser.
Now, if you don't mind, please be so good as to replace the word 'turtles' with 'time', and 'down' with 'back'.
It's time all the way back, hmm? Did you not just express the opinion that such sentiments were silly? Well, I did, but you weren't in any real hurry to disagree, were you?
That's what I thought. Run along and play, little dismisser of arguments, while we astrophysicists have fun with our special theories.
And, lo and behold, the time-all-the-way-back theoriser hangs their head in shame and departs. Which is silly, really, because when you start to play with theories about the origin of the space-time continuum, infinite time is a possibility. So is an infinite pile of turtles, for all we know. Some of those special theories get rather strange.
Naturally, in the end, the age of time is left as a question mark. The point is made that, even if time is standing on an infinite pile of more time, it still needs explaining. You ain't getting out of it that easy.