Probably depends on what position level you are talking about. I haven't done
it, but have had it happen to me and have seen it play out at places I've worked. There will be people that won't be happy, but they will probably forget about it within a week unless you accepted an executive level position. In which case you've got to be more careful as folks have longer memories and your universe is smaller.