But what you are trying to do is provide a counter-hypothesis to the data that has been presented in my original post without providing the corresponding statistical support.  However, I would consider that to be akin to saying that you can't statistically compare them.  That being said, if you want to either critique the statistical approach I have taken (e.g. why is a z-stat not the correct aproach and what would be better, is there a different null hypothesis to test, can we assume that this variable should be assumed to be normally distributed, etc.), or provide a counter-argument with statistical support, I would highly enjoy that.  

The original premise, put forth by Shoganai was that perhaps you should remove the BYU-Utah games from the sample.  That, in and of itself, appears to be cherry picking data.  However, what Shoganai is really doing is what all good analysts do---he is looking at the data, seeing something that doesn't seem to fit the pattern, then using that to develop a hypothesis.  This is where I tried to follow-up on that.  Shoganai's unstated hypothesis is that BYU's win % against Utah is meaningfully lower than BYU's win % against comparable P5 teams.

This is the hypothesis that I tested in my original post.  We run the test 3 different times with the different segments of the data: (1) All of BYU's P5 opponents; (2) BYU's opponents with winning records; and (3) BYU's opponents with losing records.  This data comes from kridnorr's original post which can be found here: https://www.cougarboard.com/board/message.html?id=25185169.

In each of these tests we are looking at BYU's win % against P5 opponents (not including Utah) vs. BYU's win % against Utah.  In all 3 cases, the statistical evidence implies that BYU performs worse against Utah than when compared against a similar group of P5 opponents.

Your argument is that you have to look at each individual game, player and set of circumstances.  Well, let's take one specific example of that which is not included in the data, but is the type of thing which you speak of.  BYU-Utah 2010.  Bradley intercepts a Utah pass and in the return the ball comes loose.  In the replay, it looks pretty clear that his knee was down before the fumble occured, yet Utah retaines possession and goes on to score the winning touchdown on that drive.  Now, if the call goes the other way does it guarantee a BYU victory?  No.  But the probability goes up mightily.  That is an example of a specific time when the data would show a BYU loss, but the further review of the game may show something different.  

And that example is exactly what the statistical analysis is showing.  The number of losses that BYU has to Utah over the last 10 years are an anomaly when compared to similar games they have played against other P5 teams.  It doesn't change the final score of all those games, but it does imply that Utah has had a few more balls bounce their way than BYU has in those games.  Luck is part of all sports, and in a close game (which rivalry games often are), an extra bounce in your team's direction can make a difference.

Neither the data (nor I) are suggesting that BYU should have won every game.  The data (and I) are simply suggesting that the probability of Utah having won 10 games in a row, even given the two teams relative strengths, is very, very low.  There was a time when BYU went 19-1 against Utah over a span of 20 years.  BYU was arguably the better team most of those years, but not 19-1 better.  I would imagine if we did the exact same analysis of that stretch of games that it would show that the probability of that happening at that time was mighty low as well (maybe even lower than the 10-0 run Utah is on).

Once again, if you want to provide a critique or suggestions on the actual methodology or execution of the analysis, or want to provide an alternate analysis, I'm all ears and would love to hear it.