That is why you invite all the conference champions — and since I don't buy into the Power 5 either, that is why it needs to be ALL FBS conference champions. It also takes much of the power away from the Power 5 if everyone has an equal path to get into the playoffs, helping the G5 teams compete on a more equal footing.

Of course, this could all be better by changing the conferences — going to smaller, more regional conferences (say 8 or 16 conferences) and, at the same time, possibly shrink FBS a bit so that you remove those schools that do not have the desire to compete at the highest level.