I've had every single one of them get hit or broken over the years. A couple of years ago when I went to fix two that had been hit by a tractor, I changed what I'm doing with my cleanouts. Instead of having them stick out above ground level, when I fix or replace one, I make the height about 1 inch below ground level. Then I fill over it completely with dirt. I have some 18" x 18" concrete blocks that I put over the top of the cleanouts to mark them. If someone happens to drive over the concrete block, it'll spread the weight enough so that it doesn't damage the cleanout. This has solved the problem of them constantly getting hit by equipment or a kid on an ATV and breaking them off.

To answer your specific question, yes you should dig it up and replace anything that is cracked or broken.