They just need inventory to fill in time slots. There are plenty of games, P5 and G5, that drew far less than 500,000 viewers.
Houston vs USF, for example drew 177,000 viewers, last year.
Last year, 36 games garnered fewer than 200,000 viewers and 14 of those had less than 100,000.
The networks still air those games, knowing in advance that they are not attractive to any but the fans of the teams playing.
Attendance has never been a factor in deciding whether a power conference invited an outsider. If it was, BYU and not UU, TCU or Louisville would be in a P5.