Someone else in earshot, the correct thing to do is to immediately let the person you called know you are listening on speaker phone. Same thing if someone walks into a room while you’re on the call.

I had a boss that used to call me and ask questions about a client or a project I was working on without telling me the client was in the room and he had the speaker phone on. While I don’t bad mouth clients and I never got caught in anything like that, I did provide more casual answers to his questions the first few times (or throw in more speculation than I would in speaking to a client). It bugged the heck out of me. After a while I learned to ask, “Am I on speaker?” every time.