I recommend you sit downs dndiscuss it openly and safely with your wife. Be ready for some pretty hard words from the wife but talk about it anyway. If you let her know how you feel she may be more understanding.
Don't be afraid to break the mold a bit. I turned things around with my ward calling and injured a few feelings but it made more time with the family and made things a lot easier and I still magnify my calling, just in a way that allows me to connect with my wife and kids better.