people call Westerners
. Of course, the term is not meant literally
, it is a habit
that stuck last century.
In Chinese lore, ghosts/demons are pale in complexion, just like as if there was no blood in their bodies. Quite different from the Western variety of spooks. When the Westerners first arrived, they were startled by the pasty color of the skin, so they were called "ghosts". Their beards and abundance of body hair also helped in the attachment of the term to the Westerners.
As the British began to force opium into China (see British Forced Opium Trade), the Chinese attitude toward foreigners got worse, as seen in the Boxer Rebellion. "Demons" then got attached to the foreigners as well.
Today, the term is used light-heartedly, never offensively. The proper, polite term for foreigner is supposed to be lao wai (Mandarin), or old outsider, but nobody ever uses it. We just say, "look, a ghost-man" or "The demon is here".