It was the fact that BYU lost almost every one on one battle in the trenches. Baylor got away with some holding for sure (most visibly on the edges), but if the DL has the size and the strength, they can get off of the jersey and shoulder pad grabbing and do some of their own.
BYU needs some bigger, faster, and stronger guys up front.