The law is that you have to carry recognized personal identification at all
times, which as an American means your passport. On the other hand, you probably won't be thrown in jail if the police stop you and ask for it.
I carry mine and my credit cards in a neck pouch under my shirt.