Oh I totally agree, and my wife and I do the same thing with our kids.
My issue growing up was that my mom's argument wasn't health reasons, it was that it's so much cheaper to buy it at the grocery store, but we never actually bought it at the grocery store afterwards. That just always bugged me.