long term, you are delusional. Having 24/25/26 year olds playing against guys in their very early 20's or younger is absolutely an advantage, and a big reason BYU is so successful this year. It's absolutely not enough to make you a better team than the other guys, but when those RMs become seniors it's a huge advantage. Imagine a team of your average NBA players playing against themselves as 19 or 20 year olds and tell me the older team wouldn't destroy the younger one. Having dudes on your team that are the same age as the average phoenix suns player is totally an advantage. They are more physically and mentally developed.
Going on a mission does set most basketball players back a few months, but you are kidding yourselves if you don't think they are playing basketball and working out every day. They aren't 100% ready to play when they get back, but it's not like those dudes are coming back 40 lbs over/under weight not having touched a ball for 2 years. It takes time for them to get back, and probably effects their performance for the next year or so, but by the time they are seniors being two years older than anyone on the other team is a big deal at that age and stage of development. You are talking about fully developed men playing against essentially high school kids.
I will say that the two years (depending on where you go and your body type) can have a pretty profoundly negative impact on your pro prospects, but for the guys that do manage to regain their athleticism/skill post mission, it is a HUGE advantage in the college game.
I'm not claiming that BYU is cheating or anything, they aren't the only school that can sign RMs, but the fact that they do sign them gives them an advantage when they have a senior heavy team.