I felt this way for awhile and sometimes feel this way. Honestly, the best thing that has happened is getting a wife who understands that I need a break. That I don't have to go to every family event. That I need some personal time every once in awhile. It goes both ways and neither of us are always perfect having the patience to say sure go spend 4 hours doing whatever, but we try. Having a young family makes it more difficult and the patience is more thin at times, but it is significantly better when we were first having kids. Felt very trapped and never able to have time to myself.