The assumption is that a team would send "cherry pickers" into their offensive third and that the opponents would respond by leaving defenders back, thus muddying the field and slowing down the game.

First, offenses would understand that, so they wouldn't employ that strategy very often. Second, the defense already has multiple defenders back nearly all the time. Not all the way back, but still. Third, leaving defenders back hurts you on offense. Sending cherry pickers forward hurts you on defense. It's a balance that teams would need to adjust to.