Simply because they are mostly there because they aren't a consensus top 25 team. It isn't uncommon to jump quite a few spots, and even jump those ahead of you even if they don't lose, because voters who previously didn't have you on their poll begin including you.

If BYU won, I think they'd move up more than two spots (of course it always depends on how upsets and everything play out). They probably have a 20-30% chance of getting in at 24 or 25. Still unlikely they crack the top, but not out of the realm of possibility.