I almost
hate to contribute to this node, but I've been thinking a lot about
rhyme lately and so it seems somewhat
appropriate.
The answer to the question about things that rhyme with orange depends entirely upon your definition of "rhyme." My dictionary (as a secondary definition) says "A word agreeing with another in terminal sound." This is the kind of rhyme we all learned in grammar school, but in fact the primary definition of rhyme is much broader: "Identity in sound of some part."
With this in mind, I encourage you to consider not only lozenge, which I believe is the closest common word to a "pure" rhyme as you're going to get, but also the whole host of other words ending in -nge, such as:
True, these do not share the -ange ending of orange, but as the A in orange is unstressed, I do not consider this a major issue. So poets, scribble away!