I think BYU can hang with P-5 programs. They have proven that. The difference is being able to hang for an entire 9 game conference season. They don't have the depth for that yet. Evidenced partly by the Washington game. Look at the toll those four games took on the team. Is BYU the only team with injuries? Absolutely not. Washington started the season without 3 of it's top WRs. Utah played part of the BYU game and two more without starting lineman. USC was missing their top DL player and a WR before Utah. They lost a starting DB during the Utah game. But they can withstand those hits to the team. Depth is what is holding BYU back from being fully P-5.
I believe the same could be said for the 2004 Utah team. They could play with anybody in the country, ONCE. If they had to play a P-5 schedule they would've never lasted the season before playing in the bowl game.