...it's not because BYU is good, it's because that team sucks.
When that same team then goes on to play games within their P5 conference, you hear "Their record might not reflect it, but that's a good football team." and "You don't get any weeks off in this conference."