I worked at the student services office at GWU and saw their grad school admissions criteria. Besides GPA, GRE and life experience points, they gave points for elite schools. BYU earned 0 points. That doesn't mean you can't get in, because I got a fellowship from my BYU degree, but it does mean that if I'd gone to a more prestigious school like Duke or Stanford, I would have gotten more points. If I'd been on the cusp with the other criteria, that may have made the difference.
People send their kids to Phillips Exeter Academy because it helps them get into an Ivy School. People go to Ivy schools because it gets them into Ivy grad schools. You have a much better shot at your dream grad school with an Ivy undergrad than with a BYU undergrad. Obviously, BYU graduates get into elite grad schools, but you can't say the odds are even with the elite undergrad degrees.
The question is: how much is that worth to you?