http://www.sports-reference.com/cfb/conferences/independent/
Look at the number of schools per year on the link above.
Click on a few of those years - just 1990 for example - 11 of those 26 teams are now in P5's (not including ND). 5 of them finished ranked #14 or above.
The numbers started going down when the rules changed on TV contracts and schools or leagues were in the negotiating seat and not the NCAA as a whole.
Then as the Bowl Alliance/Coalition/BCS started forming most abandoned independence for conferences specifically the creation of the Big East in 1991 and the expansion of the ACC/SEC...
It was more about being a fan of a team then and not bumper sticker palooza for your conference...