I get to the same point. I'm okay with disagreeing with people. It's actually a good thing to discuss disagreements.

I'm of the opinion that giving words power over you is not the smart thing to do. To be honest I do it too though. I think it's natural human psychology.

What I'd tell you or others is that a portion of the people that use that word aren't really trying to tie it to sweet persons with mental disabilities, they're just using it a more colorful synonym for bad. And the other portion that might be trying to tie it to those sweet persons are people that you shouldn't even care what they think anyway because they are morons.