Begin to fall apart? For me it happened last week when attempting to summit a difficult mountain road on a bike. I made it half way up and realized I was not going to be able to do it in the suitably ample time I had allotted.

Up until that point I had always been able to accomplish any endurance activity within a reasonably large amount of time, regardless of how much I had trained for it. It was just a question of knuckling under and gritting it out, and wondering how long it would take, not a question of if I would fail.

I'm going to bet that the answer for most people is in their mid-30s.