There are 2 theories about this phrase. The first is probably correct, and the second probably a Victorian attempt to clean it up without using the word damn.
Pots and pans used to cost a fortune back in the old days (even before my time). When one got a hole in it, it was a major disaster. So tinkers were called on to fix the things. Tinkers could have gotten their name from the sound of pots and pans tinkling when they clash, but it's most likely from the Middle English tinnkere, for tin worker.
Now, although the tinker was a needed fellow, he tended to be a
rough sort of character. Sort of like a
traveling salesman these days. (Or, just think of
moJoe.) The first theory goes that the tinkers swore so much and so loudly that their
oaths lost the power to shock. So, to not be worth a tinker's
damn was to be
worthless.
The second theory says that it's dam, not damn, and it refers to the piece of clay or whatever the tinker used to block the hole while he soldered it. When done, the dam was discarded and worthless.