I think it partially depends on the woman. My wife is half my size. I have a
hard time imagining a scenario in which I'd ever be justified in using violence against her. On the other hand, if Ronda Rousey were attacking me, I'd feel justified in doing anything I could to defend myself.