The way I see it, the lack of conference affiliation does affect attendance. Hosting the exact same opponent in October or November with the chance to win a conference championship and earn your way into a bowl game is more meaningful than hosting New Mexico in October or November with nothing on the line if we're not still undefeated.
This is not an argument to go back to the MWC by any stretch. I'm just saying, the same opponent with something to play for is better FOR ATTENDANCE than the same opponent with nothing to play for.