If for whatever random reason the conference become lopsided for a few years, the draft structure will greatly contribute to the fact that it stays that way.

Mediocre teams in the bad conference will still make the playoffs and get bad draft results. While the bad teams in the other conference get the top picks. Over a few years this will quickly create an imbalance in talent to one conference.