I had to rewatch that part to make sure I heard it correctly. At first I thought that he had to be going back to kill Cersei, but nope. With absolutely no development at all, he ditches Brienne and heads back to Cersei for no apparently reason. Maybe he had a dream.