I hate it when people tell me this.
Just the other day I was at work showing my friend Derek the webpage I had just designed for the company. "Look," I said enthusiastically, "if you click this little arrow on the front image here, the image will change!"
"You have far too much time on your hands," he said. He then added "thanks for coming out," just because, well, he always says that.
That made me mad. I didn't spazz out or anything, but I was still angry. Look, sometimes I spend time doing things that other people don't understand. Sometimes I spend time making silly JavaScript programs that randomize words, or make pictures change. Sometimes I synchronize Pink Floyd's "Echoes" with 2001: A Space Oddysey to see what it sounds/looks like. I do these things because I want to, not because I have too much time.
And besides, even if I do this stuff in my spare time, at least I'm doing something constructive. I'm not watching TV or looking for internet pr0n. I'm actually building things.
Some of the greatest revolutions in science were made by people with too much time on their hands. I mean, Einstein figured out relativity while working as a patent clerk. If he told anyone that he spent his day daydreaming about time and space, most people would think he had way too much time on his hands. Or how about Shawn Fanning. If he went around telling people he was building "MP3 sharing software" in his dorm room, they'd think he'd have too much time on his hands, and they'd also ask why he wasn't out drinking.
In conclusion, if you think someone has too much time on their hands, be nice about it. At least feign interest. Then maybe they'll share their millions when their silly projects get famous, and they rich.