when I file at the end of the year. I make more money earning interest on that amount in my account than their late payment fees for not paying quarterly.

I think it's ridiculous that they can charge me a fee for not paying income taxes in advance. But I'm never going to pay quarterly taxes. We just keep the money in a 5.25% interest bearing account, which is more than the fees end up being. Last year, my late fees were about 2% of my total tax liability.