about 9 or 10 years ago, I was shocked to see that the medical school with the highest admission standards in the entire country (in that particular year) was "Washington University of St. Louis" - which I had never even heard of before in my entire life. And exactly for the reason you mentioned: they have very limited brand recognition outside of the bio-medical sciences.

Johns Hopkins is similar, but to a lesser extent. Hopkins is apparently THE juggernaut of college lacrosse, and seems to have a strong division III football program, but overall the university is not particularly well-known on a national level, despite being one of the most prestigious universities in the world in medicine and public health (among other fields). As odd as it seems, their brand would be much stronger if they had a nationally-prominent sports program.