that the way it is used in some instances is meant to be derogatory. You can argue that any offense that should be taken should be at the intent and not the word, but you can't deny that some ways that it is used is meant to signal inferiority.
Same can be said for a lot of words, but you can't give a blanket endorsement for anytime the word is used.