Well, I'm Agnostic, not Atheist, but I use two methods. When I'm expressing strong feeling, I simply say "swear," as in "I SWEAR I'll get the money to you tomorrow!" When I'm actually committing myself to a formal oath or course of action, I tell people that they "have my word on it."
To my way of thinking, in saying that they have my word, I'm actually saying that I'm staking on that oath that person's trust in me (and the trust of the other people who might find out that I'd broken my word). Of course, I do that every time I say I'll do something, but by giving my word I'm stating to the recipient of the oath that i'm aware of what's at stake.
No person can stake anything more on an oath. Those who are religious may swear to Whomever, but in most cases they believe Whomever can forgive any sin. Thus, they are not really swearing by anything additional. To be sure, a minority of the religious have a personal love of God and do not wish to upset Him, but those, by and large, do not make oaths on their religion as a matter of principle.
When I want to express how glad I am of a chance event or stroke of luck, I say that I "thank my lucky stars." To be sure, that expression has astrological roots, but it is not likely to be mistaken for that nearly as much as "thank goodness" is seen as a euphemism for thanking God.