I took my two youngest on their first backpacking trip back in July. We went up in the High Uintas. My boys were 10 and 8 at the time. They had a blast.
I totally agree that doing this type of thing with your kids is what matters to them. Spending quality AND quantity time with dad is what makes memories for a lifetime. Something about spending time in the outdoors just makes it even better for them.