this is definitely going to drive away audiences. Stuff like this is what has led me to enjoying the NBA more and more.
They have their fair share of refereeing related problems, usually a case of players mastering the art of gaming the system and manipulating calls, but they usually take action to fix it in the off-season. (For example, the swing the arms into the defender rule, the tossing the ball in the air towards the hoop after getting bumped and getting free throws, etc.) The NFL makes rules to make the game worse, the opposite of what the NBA does.