So Anselmus's well-known 'proof' of God's existence stating that everything has a cause, therefore there must be a first cause, which is God, is indeed refuted by standard high-school mathematics (the theory of limits).

This theory establishes infinite regress without supposing infinite numbers. Indeed, we can do math without infinite numbers, with a single one, or with infinitely many. But the mere idea of infinite regress doesn't decide between them.