In order to get the G5 spot you have to go undefeated or have one loss, at least that is what history has shown with the NY6 spot. That is usually a top ten team.

That's just not true, when you look at the facts.

2014: G5 (or rather, 6th conference) invite would be Boise State (#20)
2015: Houston (#18)
2016: Western Michigan (#15)
2017: UCF (#12)
2018: UCF (#8) - actually ahead of PAC12 champ UW (#9)
2019: 12-1 Memphis (#17)
2020: 11-0 Coastal Carolina (#12)

So if by "usually a top ten team" you mean "it's happened once before in 7 tries" then we're on the same page.  And competing for the G5 bid doesn't preclude BYU from competing for the other bids, so it's not giving up 6 chances for 1 shot.  We're giving up 6 chances for 7.

More importantly though, look at the above and your own logic.  We haven't finished in the top 10 for decades.  The last time we would have made the playoff (using pre-bowl CFP/BCS/AP rankings) would be 1996, with additional appearances in 1979, 1983, 1984, and 1985.

With conference affiliation it's tough to say for sure, given the reordering of everything, but assuming one spot for the best champ outside the major conferences, BYU in a conference makes it in 1974, 1976, potentially 1978 (hard to say as no teams outside major conferences were ranked), 1979, 1980, 1981, 1983, 1984, 1985, 1989, 1990, 1992 (despite being 8-4), 1996, and 2001.  I'd also imagine that making the playoffs 12 times in a 19 year span ('74 to '92) might have done more to cement BYU as a top team.