If you exclude all the people who are fans of either team (per your instructions) and then all the people who are just fans of football and would watch any convenient playoff game, there's only a miniscule amount who are tuning in to BYU (or Utah or Texas Tech or any other non-marquee name) specifically to see THAT team play.

People watch games to see their team play, to see teams play in big games or to see 'name' teams play