Taking Rudy out would have only made things worse on defense.
It's fair to criticize Rudy for his limited offensive game. Had he been able to score over guys like Reggie Jackson, it would've forced Lue to change his approach. But on the other end of the floor, Rudy was the only player that gave the Jazz a chance.