BYU already has a home for all their non-football sports, so that wouldn't be an issue (which was one of the harder parts in leaving the MWC).
And the AAC already has 1 member for everything-but-football, so I can assume that they wouldn't need other BYU sports (or want them, given the travel issues you mentioned). So something like 4 or 5 trips a year for BYU football, and one every other year or so for the other teams doesn't seem like a deal-breaker.
Good catch though and something I should have mentioned that I envision this as football only.