Sometimes the call a game tight, sometimes loose, but I don't think that the refs calling things either way (so long as they're consistent on both sides of the ball), is an excuse for why you lose a game.
The times (like Wisconsin v ASU, or was it Arizona when the refs wouldn't set the ball) when the refs really screw over one team are rare.