The BYU admission standards have been rising steadily over recent years, and it is a relatively high bar to meet academically. It's a recruiting challenge, but not insurmountable - but it does make the job harder for sure. Coaches have to put a lot more time into recruiting (especially scouting) planning on the fact that some of the higher rated kids won't get in. It also requires finding sometimes the lower rated, lower star kids that have tremendous work ethic and high performance potential.
I remember coaches spending ridiculous amounts of hours combing over paperwork, videos, results, and information to find target recruits, and spending a lot of time on the recruiting trail, not to mention all the phone calls, follow up, visits, coordination, etc. When a 4-star got denied entrance to BYU, they went to the next on the list and found talented, hard working, and ultimately great players to fill team spots. It just takes a ton more time, I'm sure they put a lot of their "own" time into getting slots filled, it was literally almost a 24/7 deal during recruiting season.
Just one more wrinkle and element to the BYU HC job. Recruiting is a huge time commitment and a unique challenge here in multiple ways. Not only do you have to work to convince your big targets to sign, you have to have really good secondary options lined up for the ones that wont qualify, often finding those diamond in the rough type players that will be good fits. That's a skill in and of itself.
Even so, there are many other elements and IMO, BYU can get the athletes needed to beat a majority of G5 schools. It's an uphill battle vs P5, but I think BYU still can get the athletes needed to be a higher G5 level with the right scheme and system.