Our society has shifted focus from the intent of the person making a comment or doing an action to the interpretation of anyone hearing or receiving an action. In other words, “it doesn’t matter if you meant it to be offensive, it hurt my feelings so it was offensive.”
This is wrong.
I don’t know this person, but I suspect he was, in an insensitive manner, using a word to disparage the ref. I highly doubt he was trying to disparage all disabled people. If he had called the ref a “girly ref” did he mean to disparage all females and hence should any women at LES been offended? If he called the ref “blind” did he mean to offend all blind people?