While every member isn't a BYU fan, there are a lot of fans, even in the Bay Area. Even if you take 3 families multipled by the 20 wards in San Jose, that is 300 attenders. Multiply that by all of the wards in the neighboring areas and it adds up pretty fast to who would actually attend a game.
90 minutes away? Where did you live? Hollister, Monterey, Sacramento or up north in the granola/Redwood forest area?
I think Cal is on the schedule? Should have a real data point soon.