Assuming that it is just to tax one individual proportionately more than another because they have more than another (which is not a forgone conclusion, but seems to be an accepted premise for our system of taxation), I think there are three general problems with "millionaire taxes":

1. Like you said, we're often taking from the middle class (even when we're thinking that we're taxing the upper class), and many of the so called millionaire taxes are nothing more than an increased tax on the upper middle class. A tax that truly touched the 1%, and no one else, seems unlikely (and doesn't seem to have been proposed or seriously considered to date).
2. Our current system of taxation lacks any consideration of relative income. We all agree that a billion dollars is a lot of money, but it's harder to agree on whether that's true of one million (or even one hundred thousand). An annual salary of $100k allows you to live like a king almost anywhere in Kentucky, but you'd be on the low end of the bell curve in San Francisco or New York. When creating national income tax brackets, we don't account for these market-to-market variances (and I worry that a tax on the 1%, written by those in Kentucky, would capture much of the middle class along the coasts and in major metro areas).
3. The "net worth" of the 1% is rarely realized; Bezos or Musk are worth billions on the books, but much of that valuation is illiquid and could vanish tomorrow if the value of their shares dropped. We open a dangerous can of worms when we start taxing people on their paper gains. Perhaps we could tax loans that are taken out against illiquid collateral, but even that seems potentially problematic (are we going to start taxing the middle class when they take out a home equity loan?)