While the overall sample of games against P5's is relatively low in the period chosen, we can still apply basic statistical analysis to this data to see what it says.

By observation we note that BYU's win % against Utah is lower than BYU's win % against P5's that are not Utah.  And this holds true regardless of whether it is > .500 P5's, < .500 P5's or all P5's.

So, what Shoganai is suggesting in his post, is that there is something odd about BYU's performance against Utah that doesn't line up against the other available data.  We can test this by doing a simple test of two sample proportions.  In our case, p1 (proportion 1) will be the % of games BYU wins against Utah vs. p2 (proportion 2) the % of games BYU wins against non-Utah P5's.

We will do 3 separate tests: (1) All games; (2) Games vs. teams that are > .500; and (3) Games vs. teams that are > .500.  We are testing the null hypothesis that the proportion of games BYU wins against Utah is the same as the proportion of games BYU wins against non-Utah P5 teams (happy to have a longer discussion about why this is the appropriate null hypothesis if anyone is interested, but the simple answer is that is the standard convention when testing proportions).  We will calculate a simple z-score and test it against a 5% alpha level.  So, if the probability that the two proportions is the same is less than 5%, we will reject the null hypothesis and conclude that BYU's win % against Utah is statistically different from BYU's win % against non-Utah P5's.  I'll also add in the p-value so we can see how much further down the "alpha-curve" that probability is.

(1) All Games:  z-score of -2.231, which has a probability of ~3.3%, and a p-value of 0.0129.  In this case, we firmly reject the null hypothesis that BYU's win rate against Utah is the same as BYU's win % against non-Utah P5's and conclude that Utah is something different for BYU.  The p-value of 0.0129 tells us that we would continue to reject this null hypothesis down to an alpha level of ~1.3%

(2) Games vs. teams < .500: z-score of -2.213, which has a probability of ~3.4%, and a p-value of 0.0134.  In this case, we firmly reject the null hypothesis that BYU's win rate against Utah is the same as BYU's win % against non-Utah P5's (when using only teams with < .500 records) and conclude that Utah is something different for BYU.  The p-value of 0.0134 tells us that we would continue to reject this null hypothesis down to an alpha level of ~1.3%

(3) Games vs. teams > .500: z-score of -1.150, which has a probability of ~20.5%, and a p-value of 0.125.  In this case, we CANNOT reject the null hypothesis that BYU's win rate against Utah is the same as BYU's win % against non-Utah P5's and conclude that Utah is NOT something different for BYU (when Utah is an above .500 team).  The p-value of 0.125 tells us that we would continue to reject this null hypothesis down to an alpha level of ~12.5% (which is very high alpha level that would rarely be used).

So, the takeaway is that there is some evidence that BYU performs more poorly against Utah than it does against other similar P5 teams.  Both when we look at all games, and at games when BYU's competition is <.500, we see this result.  The result is not the same when we look at above .500 teams, however, while we cannot reject the hypothesis that there is no difference, the probability of winning 0 games against above .500 Utah teams when compared to how BYU fares against other above .500 P5 teams is only about 20%.  So, doesn't pass the statistical significant test, but it is still more likely than not that BYU seems to lay an egg against Utah relative to other teams.

May conclusion is that (albeit from a small sample size), there is some validity to Shoganai suggesting we look at BYU's record against P5 teams both with and without Utah included.