A good chunk of it is mental, and the brain continues to develop well into one's twenties. I think that's part of what happened to Brady and several others.
So QBs need more development after college than RBs, and this allows some lesser college quarterbacks to rise and squeeze out those who didn't continue to improve.