If the fans were civil, I would agree. The rivalry is just overboard with the fan behavior (both teams). Given that, I would prefer not playing Utah. It is just not worth it.
One other aspect that is less important than the first to me is that a Utah football game does not bring the national exposure and attention that Notre Dame does. So if I had to choose between Utah and Notre Dame, I would pick Notre Dame.
Considering on the field football only aspect, I agree with you. Utah games have been competitive and great and would be fun. The Utah team has some great coaches and players, etc..