Space is a luxury. It's even therapeutic at times. If it's only $20,000 more, which for a home is not a big deal at all. As long as you are able to stay on top of things financially with a higher mortgage and this isn't going to put you in a bind or a crunch then go for it.
As for your son, he might not see his new buddy every Sunday, but two blocks is easy. It's not like you're moving to another city or another state. That's completely different. But your son will still be friends with his buddy, unless moving also puts him in a different school. It sounds like that's not the case though.