I'm not saying they are justified, but most of those types of comments are just responding to the notion that "we need more [x nonwhite player because they are inherently better]" type of argument. Vai Sikahema was very vocal in this sort of argument, and it rubbed people the wrong way.
I'd gather than 99.99% of the CB population agrees with the notion that they don't care what color or creed the player is so long as they can play. Where you see the conversation devolve is when someone starts down the road of "but all the lame white guys from Utah county are on the team ...."