1st - the reff is just trying to keep from having to call a penalty - that is good officiating.
2nd - Where do you draw the line? What does "out of hand" mean. Does it mean in your opinion?

The problem is, the more latitude you allow before the "line" the harder it is to officiate and it becomes much more of a judgment call for the reffs. If you stop it early then the problems are far less and there is less potential for favoritism.