Work lunches aren’t much to brag about, but being left out sucks. I had a job once where a bunch of us younger people would go out and we rarely invited older colleagues. For no reason really. Probably because we just wanted to relax and talk about sports and inviting people we didn’t know as well made it too stressful for a work day lunch.
If they didn’t like you, they’d probably just go out without trying to hide it. They probably just don’t want to hurt feelings.
I’m one who values personal relationships, even work ones, so I get what you’re saying. Not sure why so many are dismissive here. Bunch of tough guys.