For one, only the "true" class matters, which includes a bunch of high caliber kids who didn't play last year. And with so many leaving on missions, it matters much more how BYU can fill those spots. They will have good talent in the program immediately, rather than sending those guys off first, and they also add high caliber guys to the incoming/eligible list like Hoge and Kirby. Not to mention we can still add Collie and Watson to hold us over for a year.
Right now it's not a huge deal. Most of the top talent on the team was young enough that it didn't create such a need. If you're graduating a bunch of top talent, that's one thing. But they return most of it. Everything is relative. He's right that it only really comes to hurt you if you don't make up for it somewhere down the line, IOW if you have multiple low end classes in a row.