In certain older Postal spellings of Chinese words, there is an h at the ends of syllables where we don't use h any more. For example:
Taipeh                         for       Taipei (Pinyin Taibei)
Moh Tzu                       for       Mo Tzu (Mozi)
Hupeh                         for       Hupei (Hubei)
Tao Teh Ching           for       Tao Te Ching (Daodejing)
Fuhkien                       for       Fukien (Fujian)
Koh (family name)     for       Kuo (Guo)
and so on. Why does that h appear?

That h stands for a glottal stop, found in the "entering tone" of Nanking Mandarin, which was the national koine in China for hundreds of years before Peking/Beijing Mandarin emerged as the de facto standard in the mid- to late-19th century. Peking Mandarin has no entering tone, but most southern dialects do, and Nanking Mandarin did, too. The glottal stop ending is common in entering tone words. Western missionaries who first created spelling systems for these languages used h to stand for the glottal stop, at least from the time of Francisco Varo in the 17th century until Carstairs Douglas in the 1860's.

All of the words in the list above (peh "north", teh "virtue", fuh "good luck", koh ) belong to the entering tone category. Final h is not supposed to be used in either Wade-Giles or Pinyin romanizations, but in a few proper nouns it survives into English as a remnant of the older spelling systems.

Isn't alphabetic writing wonderful?

Hmm... now that I think about what I said about Wade-Giles...

Actually, Wade-Giles romanization does use final h, in three finals: -ieh ~ yeh, -üeh ~ yüeh, and -ih. It happens that the majority of syllables using those endings are of entering-tone origin. The -h also serves to distinguish them from plain -e and -i.