5th is your personal responsibility.
Something I found interesting is that you generally do need to say "I am not answering questions." Or "I'm invoking my 5th amendment right to remain silent."
Simply not talking isn't enough. You must proactively invoke the right.