My take on the situation:

No matter what you do, unless you go out of your way to avoid it, you WILL touch some dirty bathroom fixture before you leave.

People go to the washroom.. then, they turn on the taps with their dirty hands, wash them, and then touch the dirty taps again. They touch the lever on the paper towel dispenser with their dirty hands (unless it's a dryer, some of which have buttons to press), and then the handle of the door on the way out.

You can't win.

I like to make a game of it. Try as hard as I can to not touch anything, turn taps on and off with paper towels, have someone else touch them for me. Sometimes I even wash the taps! It's kind of fun, and you can involve others in your little game as well.

Hint: try opening the door with the sleeve of your jacket over your hand.