...never had a single Vietnamese person misunderstand or correct my pronunciation of pho.

The tones (rising and/or falling voice) were the thing that was hard to get used to when learning Vietnamese.

And since the same word with a different tone had a completely different meaning, it's important to get those right when speaking and sometimes darned near impossible to differentiate when a native speaker is talking to you.