It is also rude to check your watch when someone is talking.
At church, you can get around that when you are following along with the TFOT lesson (or reading scriptures) on your phone. The phone is already out for a legitimate reason, and the time is displayed at the top of the screen, so I can clock-watch without the teacher knowing.
Unless we are already past the time to get out, in which case I would probably check my watch very indiscretely in an effort to send the message that it's time to be done.