If you feel like you should go - then you should go.
You might only be there for 5 minutes, and it might just be long enough to give a hug and say you feel their loss and are hurting with them. But you feeling like you should go is enough for me to say you should do it.