First of all, the number of bowl games being played in 1984 were minuscule compared to today, so your point on bowl teams played is difficult to make. In addition, BYU playing Michigan did suck. But you know how bowl tie ins are, and they were much worse in 1984.
Next, you asked what BYU did that Utah didn't...it's easy, the answer is body of work. BYU had already been making noise. The voters had faith that what they were seeing, and what BYU was doing, was legit because they were obviously not a one year wonder. I submit to you that if Utah had been more successful from 2005-2007, that Utah very well may have been in the nc game, rather than playing a bama team that wanted nothing to do with any game other than the nc game.
This is why BYU didn't really fall into the top spot. They didn't have to vote them number one, but they did. We have seen many undefeated teams not even sniff the nc, much less the top five, yet BYU DID win it. They DID get voted in. Remember, the two teams in the 2008 nc game were not undefeated.