It's worth playing them because of the history/geography/emotions/bad blood between fans, or whatever else, that makes them unique in the first place. Most of the ones I mentioned are in-state, which creates an obvious rivalry situation.
Rivalries can't just be forced (i.e., Yewt nation trying to create Colorado as a rival) -- they need time to marinate, stew, create themselves because of significant loss/win, etc., over time and then they can become something special and unique. If Utah wins the PAC-12 South the next 10 years and CU is in last every year, there will be no rivalry.
Utah can't make the Colorado game a rivalry anymore than we can just point a figure (in the past) and make New Mexico our rival. Real rivalries can't be manufactured like that -- they take time, happenstance, sometimes they come out of nowhere, and they are great when they turn into something important/significant.
The fact that BYU are in the same state, exist 40 miles apart, and are already one of the fiercest rivalries in the country are substantial-enough factors that this particular rivalry will never die, whether or not it remains "primary" or not.
GO COUGS.