Generally when all you do is attack something you eventually come to disdain it. This is talked about at great length in relationship books. If you compliment your wife a lot and always are kind to her then you will have a great marriage. If you constantly find fault then you eventually come to despise your wife. It is a natural thing and given your reaction to your family wearing BYU gear it seems probable that your constant negativity to BYU is having a very natural effect on your perceptions.

Ill bet you have a lot more red in your wardrobe than you did a few years ago.