He obviously has never hated the fans, though he does hate some of the behaviors of (at least some) fans.

I think the question is if he resents being accountable to the fans, or If he dislikes many of his responsibilities which are there solely to keep fans happy, or perhaps if he hates the loss of privacy. There are a few other possible along these same lines.