My dad is a Utah alum and so I grew up going to (and watching) Utah games with him. My younger brother was a BYU fan (just based on the church affiliation and his childhood friends around him liking BYU) and so those games were always fun. I rooted fully for Utah and had a great time doing so. Utah even started to do better when I was still young. I was one of those on the field that Lenny Gomes said would be pumping his gas.
So I was a full Utah fan... but when it came time for college, I never really considered Utah. I was honestly looking mostly at out of state schools until BYU gave me a scholarship offer that I just couldn't turn down. For those who've gone to BYU, you probably recall that a huge part of freshman year is the All Sports Pass and going to games with friends. I wasn't a BYU fan initially... but there's only so long you can go to every game with your friends and have a great time at games for a team representing the school you attend until you grow into being a fan. That was only solidified when I later married a BYU grad and watching those games with my own kids. (Though, BYU was pretty terrible the years I was there: 5-7 and then 4-8 under Crowton)
Interestingly my brother (who was a BYU fan as a child) no longer is. He attended neither school, but his wife is a Utah grad and so now (despite living in Provo) he roots strongly against BYU.
As to what would turn me against BYU? Hard to say. Would have to likely be a complete falling out between me and the Church or something horrific. Like if my kid went there and there was a Joe Paterno / Penn State situation that happened to them, that would obviously make it impossible. But nothing that's realistically very likely