Your board, do as you see fit & most of all keep your sanity. I've moderated on
some other forums and it is a thankless job.
Some people don't appreciate this, but the healthiest communities are the ones where the rules are stated and enforced. An online community this large this can quickly descend into 4chan.