Because while the committee is independent of the bowls, it does not seem to be independent of the P5, which is potentially worse. I can see, particularly this year, BYU downgraded based on SoS (didn't play enough P5 teams).
Regardless, my point stands. We have no guarantees, we are at the mercy of the committee who, from what we know at this moment, will give clear preference to P5 teams (SoS).