Here's the only interesting seasons
2001: BYU makes it as the the highest ranked conf champ besides the current P5. Overall rank (AP) #19, so wouldn't make it as at-large though
2006: BYU has an overall rank (BCS) of #20. Given Boise is #8, BYU doesn't make it as champ or as at-large
2007: BYU has an overall rank (BCS) of #17. Given Hawaii is #10, BYU doesn't make it with conf champ or at-large spot
2007: BYU has an overall rank (BCS) of #16. BYU is not the conference champ (Utah) or a top 12 team, so no playoff bid either way.
2020: BYU is not in a conference, so no championship, but finished #16 in the CFP ranks and behind 2 conference champs, so no playoff bid either way.

So rerunning the 23 years of the modern era of college football under the two scenarios,
BYU makes it to the playoffs once (2001) as the 6th conference champion.
BYU never makes the playoffs as an at-large team.