Many people who know I'm Mormon assume I'm from Utah, even though I'm not. They are ignorant. I've had to address this issue a few times in my career and I have a buddy who brought up this issue when he was applying to law firms out of law school (i.e., firms in Texas or wherever being concerned about investing in a Mormon who is likely to go back to Utah).
I agree that it's not just a Mormon thing, but it's an added layer because people so closely associate Mormons with Utah.