nor should they be. The reffs are there to enforce the rules and keep the game in control. Sometimes grabbing a player is required to do your job. On the other hand there is never a reason for a player to touch a reff unless it is to get their attention - like tugging on a sleeve.
Reffs perform the same function in a game that police do in society. Do you think there is never a reason for police to touch law breaking individuals? Watch an NHL game and tell me that the reffs are out of bounds when they are breaking up a fight.
There is only a double standard if the two sides (players and reffs) have the same standing on the field. They don't.