There are a few very important facts that will keep you sane.
One: the avenues with names do not run continuously parallel to the numbered ones. In other words, they cross each other. Broadway's a major violator of the grid-rule: it runs east of 5th avenue below 23rd street and west of it above. It crosses Amsterdam at 72nd street. It jumps a block east at the bottom of Union Square. If you're lost, don't say 'oh, look; Broadway' and think that everything's alright. It's not.
Exception: Some avenues have nicknames. Nobody uses these except tourists. 7th is Fashion Avenue, 6th is the Avenue of the Americas. These are considered numbered streets.
Two: The grid rule will screw you royally below 4th street. Manhattan starts bending to the east around 34th street or so, but it's not really a problem till you hit the Village (correction thanks to chris-o). The avenues start to do crazy things, like slooooowly bend so that you could be walking down 6th avenue and run into 7th avenue as a cross street. 3rd Avenue changes into Bowery at Cooper Union with nary a warning. With that in mind, it's still better to stick to the numbered aves down there (or for that matter, anywhere) - worse comes to worst you can walk back uptown til things even out a bit and figure it out from there. Which leads me to:
Three: The Empire State Building is your friend. Look for it. If you know you're downtown and you see it to your left, left is uptown (north). If you're uptown and see it to your left, left is downtown (south). If you're in midtown you should be okay; the grid rule applies.
Four: Remember this: 5th, Madison, Park, Lex, 3rd. That's the order of the east side avenues for a good portion of the island, west to east. The corresponding west side avenues are B'way, Amsterdam, Columbus, CPW (Central Park West), but because of Rule One it doesn't help too much.
Five: As far as you're concerned, there is no 4th avenue. Ok, there is one, but it's not worth looking for. Trust me - 4th avenue is what Park Avenue South turns into at the bottom of Union Square at 14th Street. It exists as 4th Avenue all the way to Astor Place ('all the way' being about 6 blocks) where it turns into Cooper Square for a block until it merges with 3rd avenue, er, I mean...Bowery. See what I mean? Stay away from 4th Ave.
Six: Ask a cop. Contrary to what NYPD Blue tells you, most New York cops are perfectly nice - if you're stuck in a really shady area you're better off asking one of the boys in blue than a stranger. If you feel safe, go ahead and ask that random guy where on god's green earth you are; Most New Yorkers will be more than happy to show you how smart they are by helping you out.
I know it seems like a lot, but trust me, remember these and you'll be fine. Worst-case senario, find a subway station and check the map. MTA workers are equally helpful. Try to find one outside the token booths so you can hear them.