So using one word is no bueno but using a potentially equally offensive word is okay because society? Or because you don't deem that word to be as bad as the other.

This is why policing language is a fool's errand.

You could also make a much easier argument that slurs related to mental illness are significantly more potent despite "societal baggage" due to the stigma surrounding it, the volume of people affected, etc.

But again, fool's errand. Sticks and stones. If someone turns me off with their language, I simply do not engage with them.