The refs are not employees of a conference, they are paid as independent contractors and all have day jobs outside of officiating. Many work for specific contractors because of the compensation, but it is not enough to allow them to quit their day job. NFL pays well enough to live off, but isn't life altering type money, so most continue to have a day job. Things won't change unless the NCAA steps up and nationalizes the officiating (like the NFL), but they won't because they do not want to increase the costs of doing business.

Your example of a juror is not very good because an official does not determine the outcome of a game in the way a juror does. Of course if you take it further, the juror is paid by the government and in most cases the government represents one side of the case...