You feel obligated to go there all the time
They are a ton of work
Most people just use it like a second home instead of a place to facilitate outdoor activities
depending on where it is located you are stuck there

Having a cabin could still be right for you and your family but my inlaws have a cabin and it is fun 1 or 2 times a year. Most of the we just sit around like it is their house and dink around on our phones.