For the first little bit, even a small pool would help you get in better shape; however, after a few weeks there wouldn't be much benefit. Even after a month (maybe a few) of consistent swimming in a 25yd pool, you get more efficient in the water and can get to the point where you can use your push to get you 1/3 or more of the way across the pool. Even after that, your momentum makes the stroke easy in a 25yd pool.

In a 50 meter pool, somewhere around the middle of the pool any momentum you've gotten from your push off the wall is gone and you're left to get the rest of the way by your own power. It is a completely different experience.

Think of it this way, using a 25 yard pool (or smaller) would be the equivalent of sprinting 25 yards, then walking 10, alternatively for a mile. If you're out of shape, this sort of workout will be helpful, but if you want to improve beyond a certain point, eventually you need to run longer unbroken distances.