There is not even close to enough data to make that determination. 12 games per team is not enough - not even close. When has there ever been a clear cut 4 or 8 or 16 best teams? Never. What you are suggesting is gymnastics - or better yet, a beauty contest. Someone is using their own opinion to determine who is most deserving - which is biased and exclusionary. Not only that, but there is nothing performance based to say that the #5 team is not good enough to win it all, just an arbitrary selections based on an inaccurate SOS and the opinions of a small group of people.
What you are saying is proof positive that the concept wouldn't work. You are insisting that the SEC is the best just because you say so and because they won a few games that you think are important. Others would think differently. Its arbitrary and exclusionary.
Well, taking conference champs eliminates the need for anything that is arbitrary except the seeding process. Prove yourself in conference and you are in. If you don't win your conference then you have proven you aren't worthy to be there. It has already been determined by conference play that you are not better than the team that won. This would make the conference season massively important and interesting.