Ok, my turn. Washing your hands is unnatural. Now I mean that, but I mean a lot of things. Hell, wearing shoes is unnatural. See my point? No? Let me elaborate:

If you can't trust your own genitalia to be clean, how can you trust the door handles, flushing handles, soap handles, paper towel handles, buttons, knobs, whatever? How can you trust that the zipper and button on your pants are clean? And the water (yes, I'm being excessive) How can you trust that anything is any cleaner than your genitalia? You can't. Don't bother.

In addition, post-toilet handwashing is paranoia on a grand scale. People are way too scared of the little things living on them. They're too scared of those things, despite the fact that you depend on those little things to survive. Oh sure, you don't depend on shit bacteria to survive, but you are naturally equipped with anti-shit bacteria bacteria. I promise. People haven't always washed their hands. Hell, a huge chunk of the world still doesn't use toilet paper (or water).

So I give you a choice. You can use the time, soap, water, and paper cleaning your hands every time you go to the bathroom to feed your paranoid urges, or you can just walk out and hope that some asshole hasn't left the door handle wet.