since most everybody voting is basically neutral towards independents, it's a team recognition vote.
everybody knows ND and many hate them for their easy access to high rankings they don't deserve. that leaves just BYU and two service academies. nobody hates Army except Navy and vice versa, so BYU gets the second most hates easily (mostly from our past conf. opponents I imagine).
same thing in reverse. ND still has by far the most recognition and many people like to be associated with a historical winner like ND. People just like to support the military so Army and Navy get likes just for being military. Navy gets more likes b/c they win more games than Army, go to bowl games, etc. so they have more recognition among neutral voters.