I would stay over at least one night to be able to appreciate the place, if possible.

Of the other three, I would go with Segovia first, then El Escorial, then Avila as daytrips from Madrid. Note that Avila is easy to hit on the way to Salamanca if you drive there; we stopped there while driving from Toledo to Salamanca last time we were there, IIRC.

Another awesome Spanish town to have on your daytrip-from-Madrid list is Cuenca. It’s a bit more remote, but awesome and somewhat austere and not very touristy, in a very scenic bit of countryside.

If you’re in the mood for castles, consider Castillo de Coca in Segovia province as another day trip candidate. The surrounding countryside is also very pretty.