If the fan base defects, it will have a long lasting detrimental affect on the program and the perception of the program. I don't think the administration is stupid enough to let that happen.
Just losing a few games on it's own isn't going to get Bronco fired...but a big enough fan base defection would without question.