Why is sexist or racist language taboo but derogatory language against the poor is fair game ?

Blame the hidden injury of class.

Just like for physics it took Newton to discover the obvious (gravity is why things fall down), Sennet and Cobb discovered for sociology the reason why poor people feel bad. Poverty is based on relative comparisions, not absolute values, beyond a point where everybody is still housed, fed, medicated and educated at a reasonable level. In meritocratic societies, affluence is associated with virtue, while the poor are considered as dangerous, lazy, dreary and unattractive carnies deserving of their fate. This makes it harder for the poor to get ahead in societies where supposively barriers between the serfs and masters were broken centuries ago. Hence in this way meritocrity is a myth.

The hidden injury of class also makes it difficult for a person in one class to trust people in another class, or to take a course of action that is beneficial for themselves if it means somebody else is going to be better off.

At worse, the hidden injury of class can be projected into more odious attitudes and behaviour. Ethnic groups who are perceived to be wealthier can become targets for recrimination (eg: Jews in Europe, Chinese in Indonesia, Indians in Africa)