down the line of scrimmage indicating that the receiver is on the line, or his backfield arm straight backwards down the sideline indicating that the farthest outside receiver is off the line, in the backfield. It's his job. Another one (I don't remember if it is the referee or back judge) is then counting men in the backfield based off of what the line judges are telling them as to who is on the line or not.

Bottom line, that one never really just slips through the cracks. Kind of like an easy false start on an o-lineman.