...people who are carpooling (IE: multiple people in the vehicle) when traffic is congested and they can't go their speed of choice in the other lanes.
If the normal freeway travel lanes are going the speed a person wants to drive - there is no need for them to be in the carpool lane. Go drive your chosen speed along with everyone else who is driving your chosen speed in a different lane.
I also believe that the carpool lane is not a passing lane and not a super fast lane. It's a carpool lane. So - I don't hold it against people who are in the carpool lane going the speed limit (or 5 over, more often) when the rest of the lanes are congested and driving at slower speeds. I don't feel like the person is responsible to pull over for faster traffic. It isn't a passing lane - it's a carpool lane.
But - back to my original point - if they are in the carpool lane, and there are others behind them also in the carpool lane who would like to go faster, AND traffic in the other lanes is moving at the speed the person backing up traffic in the carpool lane is going - then get out! You don't need the carpool lane to go your chosen speed - so let others use it to go theirs.