Walmart can destroy all of the small business in a town. Amazon won't.
People still like to run to the store to get groceries, hardware, etc. Walmart tries to be a one stop destination for everything---groceries, clothing, hardware, etc. Walmart has much more potential to destroy all of the local small businesses.
Also, Walmart is notorious for leaving a town after a few years, and after it has gutted the local economy.