This is a song that haunts.

It's not the lyrics: They are rather innocuous when compared to other ditties from 69 Love Songs.

It's not the vocals: Stephin Merrit's melancholic bass is moving, but isn't what causes the song to get under the skin.

No, it's the music: At the start of the song, the piano rumbles down on the lower range, restlessly repetitive like crashing waves, reflecting the despondancy of the lyrics. Then on the phrase
Hey, Lady Day,
as the lyrics become more hopeful, the piano moves gracefully up into the middle range, reaching up towards the top range and touching it before dropping back to the middle. But on the second
Hey, Lady Day
the piano reaches that top range and flits lightly and almost carelessly, just as the singer has reached an apex of optimism. And then, when the singer realizes how foolish his optimism is, the lyric stops just a few seconds before the piano, as though the instrument has taken on a life of its own.

There is a fraction of a pause, silence suddenly intruding...

...before the singer drops back down to his original bleakness, the piano a split-second behind the entry, almost unwilling to return to the restlessness of the lower range after it's joyous romp at the top range.

And then it happens again: the singer doubles back to the first "Hey, Lady Day" lyric, expressing his optimism once again. Once again, when he sings
Will you be my only friend
and breaks off the lyric, the piano continues for a second or two on the top range before breaking off as well. And there is sudden silence.

Only it lasts this time.