The first firm was larger and more prestigious than the second.
I lasted at the first one for almost 3 years. My leaving was more a function of my specific situation at the firm at that time than anything else. Specifically, I'd been billing most of my time to a single patent litigation for the better part of the previous 2 years. When that case ended, my billable hours dried up. Desperate for work, I reluctantly agreed to join another litigation team that was headed by the most abusive partner in the firm. About a month later, I couldn't take it anymore and started looking for another job.
My second firm had a more humane work environment (the managing partner was actually a super nice guy) and I lasted there for a little over 4 years. In that case, I just reached a point where I became tired of the DC law firm lifestyle, in particular, the super long days that were the natural consequence of trying to get enough billable hours combined with a long commute (and almost no vacation time). Eventually, that lifestyle just took its toll on me.
I ended up getting an in-house job with a Fortune 500 company headquartered in a small town in upstate NY (my total daily round trip commute time went from over 2 hours to less than 20 minutes and no more billable hours). Overall, my life quality was quite a bit better than it was in DC but there were other factors about that situation that were more difficult than I expected (for example, I struggled with balancing work and church responsibilites).
I now work remotely in Utah for that same company. My work-life balance is better than its ever been but I'm becoming increasingly burned out of my career.