A nonce word is a word that a writer (or speaker) invents to fill a very specific and pressing need. They are, in a sense, throw-away words, words that are not expected to ever enter a dictionary; words for the nonce. English being English, however, they often do make it into common usage, or at least into dictionaries -- or failing that, we collect them here on E2.
There are a few different categories of nonce words, however it is generally true that they should be recognizable to the audience with little or no explanation. As such, they are most often some form of compound word that a sufficiently literate person can decode without too much effort -- although it may require some reliance on context cues.
The most common method of creating nonce words, and the easiest on the reader, is to use familiar word parts in new ways; from the obsolete hexagonally to the modern truthiness, wordsmiths have always created new words by combining old words with old pieces in new ways. Edgar Allen Poe gave us the ever-popular tintinnabulation, H.P. Lovecraft gave us the rather less useful hippocephalic, and Snoop Doggy Dogg gave us bootylicious. Of course, many more mundane words are created this way than are gems like those listed above. Unputdownable, besmitten, offshoring... and others that were more truly nonce words, that have slipped into easy disuse; roundaboutation, ideolexicalization, sphallolalia, skyhookery, stelliscript, stupiditarian, floccinaucinihilipilification, and tetrapyloctomy1.
Lewis Carroll was a master of the portmanteau word, in which two familiar words are conjoined and contracted in an unfamiliar manner. He give us galumph, frumious, chortle, mimsy and slithy2, among others. It can be quite hard to determine the specific meanings of portmanteau words; they do work wonderfully in poetry, though, giving the author full control of meter and rhyme.
Edward Lear used a related method to create words, distorting common pronunciations of awkward words, and giving us runcible and bombilious. We get a lot of slang this way. Patootie, cockamamie, bubba, blimey, and zounds3 were all created by distorting common words, as were the more splendicious monstrosities intertwingularity and supercalifragilisticexpialidocious. Minced oaths are nearly all constructions of this sort from heck to fudge.
Neologizationating is not a new hobby by any means. Shakespeare appears to have created words such as omittance, swagger, foulmouthed, and footfall. We don't know for certain that Shakespeare was the first to use these, as many words were in circulation in those times without necessarily making it into print, but it seems likely that along with his poetic creation of phrases such as 'one fell swoop' and 'bated breath' he also created a number of nonce words.
The term 'nonce word' might almost itself be considered a 'nonce phrase', coined by James Murray, one of the more influential editors of the Oxford English Dictionary. Before he defined this specific flavor of hypothetically temporary word creation, the phrases neologism or nonsense word were used (and they still are today).
For obvious reasons, all of my examples are words that caught on... at least a little bit. Nonce words that do not catch on at all may be referred to hapax legomena.
1. circumlocution, neologization, coquetry, magic, writing in the stars, stupid, trivializing, splitting a hair four ways.
2. gallop triumphantly, fuming furiously, a chuckling snort, miserable and flimsy, lithe and slimy.
3. (Sweet) potato, decalcomania, brother, blind me, God's wounds.