50 is definitely more than enough for 1-2 devices. Who's your ISP?
Are you renting a modem, or are you using your own modem? If any of your hardware (e.g., the modem or a router) is really old at any point in the chain, it may be causing your slowdowns.