in addition to the ones listed by agentgray in all the fantastic advantages an invisible man would have in the world:
he would be blind. Yes, blind. As in, "unable to see." If he and thus his eyes are invisible, how is light going to be absorbed by them, allowing him to see anything?
No one could be careful not to injure him, because they can't see him. the truck that would ordinarily yield to him in the crosswalk doesn't know that he is there.1 So he has to be really careful, all the time...
He would have to be naked, to be truly invisible. Well, to be fair, that's a plus and a minus; nudists would probably think that that was pretty neat, but by and large, clothes are a good thing.
He would have to stay really, really clean, or a (visible) layer of dust and grime would start to form on his skin.
AND HE WOULD BE BLIND, MAN! But's that's the price you pay, to be an invisible man, I guess...
Yes, I did, in fact
that example from an episode of The X-Files