You will undoubtedly offend people by not accepting friend requests, that's just the world we live in now. I wouldn't answer any personal instant messages from any coworker though.
In the future if someone leaves and you want to unfriend them, then by all means.