The institution and its top employees (and most boosters) not being willing to bend the NCAA rules (i.e. cheat).
The institution and its top employees (and most boosters) not being willing to overlook serious honor code violations (anymore).
Being affiliated with the LDS church.
Being located in the intermountain west.
Having leaders and fans (like myself) who are willing to spend a certain amount on football, but not as much as "elite" universities and fans.
I'm not saying these are all negative, I'm saying the state of BYU football is a direct result of these things.