I have and will go to every home game and some away games. But when you have 3-4 years of bad, I think it is typical of the less fanatical to become bitter or disinterested for any number of reasons.
The Heaps/Nelson/Doman era soured a lot of fans. I'm sure it will take several years of being good to get them back or replace them.