Before having kids, I went to every game. Not a problem. Now with two, there's no way I can justify dedicating 6-7 hours on a Saturday to go to a game and leave the kids with my wife. Most things we need to get done during the week have to be put off because there's just not enough time to do it while I'm at work and the kids are running around all the time.
Finances isn't an issue. My dad buys season tickets for all us kids every year, and this year I told him not to because it just isn't responsible for our situation for me to be gone that long on a Saturday.
And after going to every home game for the last 8 years, I feel justified in not wanting to sacrifice to go watch them play cupcakes at home... at 8pm...in the cold.