I think the church is getting the most economically efficient benefit from BYU football. I've never bought the idea that BYU winning games results in any baptisms, but I do think it gets the name of the church out there and proves to the world that we're not all weirdos. Competing at a G5 level accomplishes this goal and costs the church nothing. The OP is spot on: there would only be a marginal benefit gained for the massive amount of money it would take to compete with P5 money. And what would that money buy? Who are the coaches who would be at BYU right now but for the lower salary? (Please don't say Andy Reid.) When the GA's look at adding $10M/year for BYU football in exchange for an overstated missionary benefit, and compare that to virtually any other way to spend that money on the missionary program, the choice is probably pretty obvious.
And all the people who say "fund the football team to compete with the P5 or shut it down" are just silly. Should Houston, Boise State, San Diego State, etc. just shut down because they're not getting P5 money?