I guess I'm saying I can see where he's coming from - that most of us have biases. Call it racism. Call it being ethnocentric. Whatever. But if you're aware of those biases - then you can control for them and, as you say, not let them cloud your decision-making. But that doesn't necessarily mean you aren't racist. It just means that you recognize it and control for it.
To be clear - I'm not saying that you ARE racist and just don't know it. I'm simply saying that someone (you or anyone else) could be racist without allowing those inclinations to affect their decision-making.