Lots use it because it's just a word to them and that doesn't make them a bad person or ignorant. Maybe naive but it is what it is. I am the type that tries not to be overly opinionated about things. I feel like it just causes a lot of contention when people feel the need to be that way. No matter what the topic is. Now if someone says something and it really affects someone I'm with, that's a different story. But even then it's hard in a situation like this. I always error on the side of avoiding confrontation. If possible. I think you handled it fine.

I know some that are super offended by swearing in any form and constantly tell people around them to watch their mouth. That's annoys me. I know that's not on the level of what you described but I think it's along the same lines. Now if he was saying it over and over again, I think that's when someone should maybe let them know. I think you handled it good.