There is apparently a rather dark explanation for some of the older TMBG lyrics. I had the strange fortune of working for John Flansburg's brother (who, for reasons not related to his brother's success, changed his name entirely years ago; I won't mention it here).

He hated his brother's band and its music when I met him (in late 1993). At first I couldn't squeeze the reason out of him (the whole matter bugged me, because I LOVED TMBG, and was secretly thrilled to be working with a brother of a John), but eventually he spoke out.

It turned out that several TMBG lyrics from the early period (don't know about contemporary lyrics, as I now live in East Europe and can't keep up on their discography) were apparently references to JF's brother, and not very lauding ones. For example, back in the days when it looked like John's brother, not John, would be the big success in the family, John referred to his brother's nickname, "Chesspiece Face":

"Whatever happened to Chesspiece Face?
There go I, but for my face
All I know could be replaced
By the facts in the life of Chesspiece Face"

Frankly, that doesn't seem that scandalous to me, but X. Flansburgh (to give him a name) apparently felt this was bitter, and undeserved, criticism of his success in that period of his life.

But more clearly critical were the lyrics of "Hot Cha", which described the period in which John F.'s brother completely dropped his yuppie lifestyle, did a 180, and became an adventurer and a radical activist. X. definitely did NOT feel that Hot Cha was about Jesus Christ (see Hot Cha). Assuming the song expresses John's viewpoint on X. at that time (which X. felt that it did), then he saw X. as an irresponsible playboy:

First time Hot Cha went away
A floating island was his home
Then the phone rang off the hook
And Hot Cha had to come back home
    -- thanks to Devolver42

Keep in mind here that one of X.'s first big travels after his "180" was "sailboat hitchhiking" and Pacific travel in general. Like all adventures, it had to end; X. apparently meant to imply that his brother was here implying that his travel ended the end of a pipe dream.

This theme of irresponsibility is presented even more strongly, but more surreally (I don't remember being told of a real-life event it would hook to) in another verse:

Second time he went away
Left the bathtub running over
Stereo on and cooking bacon
Never came back to tell us why

The song expresses strongly mixed feelings and confusing jealousy (which I would forgive John, as he states it quite openly in the song, but which X. could not, at least when I knew him, forgive):
If that honey would come back
We would throw such a party
Drink and cook the prodigal son
Fondue forks for everybody

Other songs may have referred to X., but those were the two that really got to him. One final note: I remember one more song with quite down-to-earth "secret references" noted by X. It's "Birdhouse in Your Soul" -- the song was at the very least inspired by a little birdhouse-shaped nightlight in the house (apartment?) where the two grew up.