PAC fans, generally, don't support their teams. Sure, USC, Oregon, and Washington will show up. But, even Arizona, Arizona State, UCLA can't fill their "larger" stadiums. Neither can OSU and Wazzu with their pint-sized 35,000 seat stadiums. And let's not get started with Stanford and Cal.
Were it not for the fact that 1) the PAC is considered (erroneously) to be a P5 conference, and 2) Utah is in the PAC, NO ONE on CB would want BYU to be in the PAC.