It basically comes down to three criteria:
1-Did you beat the teams BYU should beat?
2-Did you win your fair share of tossup games?
3-Did you get an upset or two?
The number of games that falls into each of these categories varies from year to year in Independence. And the number of P5s on the schedule is irrelevant when looking at the schedule in this way.
BYU didn't lose to Toledo, USF, SDSU, and Hawaii because of injuries, etc. They lost because they didn't play well enough. They had plenty of talent to win all of those games. The coaches didn't get the players properly prepared and the players weren't ready to play.