The two things that people commonly (used to) notice when upgrading their
systems while in X are
sawfish and
xfs (or any other font server, like
xfs-xtt).
Sawfish users need to run sawfish-client -f restart each time there's a new upstream release.
This is often the case with zsh, too (the completion subsystem starts spewing errors).
People who use xfs need to restart X after upgrading because in some strange circumstances, their fonts can get really screwed up, you only see squares instead of characters in some applications.
Apparently it's also sometimes necessary to restart X with some USB mice -- according to avalyn.
So, the general answer is "Yes, if you're not upgrading a production system". :)
Why the downvote? There's nothing but truth here. Get the whole node removed if you think it's useless.