Just pass as quickly as you can, then get over and go about your life. No time for creating drama on the road, even if you weren't the one who started it.
Following the law to the T means passing in the left hand lane - then getting over to the right lane when not passing. When you do what you described, you are also breaking the law and making the roads more dangerous.
Be the bigger man. Do your part. Keep the road as safe as possible for everyone.