As you said, Coach Rose has a guard and perimeter oriented system. Oh, by the way so does just about every other team that plays basketball right now at any level. This year, coming in BYU really only had 4 guards with true high level talent for 120 minutes per game. 2 of them picked up significant injuries in the pre-season. LJ Rose was never right. Bryant took a long while to be truly healthy. Then TJ Haws was just off a mission and very inconsistent, mistake prone and not anywhere near the physical strength or presence needed for where BYU wants to be.
Talent was the issue. You can argue that Toolson or Chatman could have changed that. You can also argue that Coach Rose was to blame for them leaving. Or, you could say that there were other contributing factors that were completely out of Rose's control.
Coach Rose dominates with inferior talent. As much as we all love our white, Utah County all-stars, objectively, BYU just doesn't have the athletes that belong in the Sweet 16 or Elite 8. They can be, but it will be on the coaching staff to make them overachieve, just like Coach Rose did with Jimmer or Tyler Haws.