Not every track athlete is destined for stardom in every sport, because track is highly specialized in ATHLETICISM, not skill.
Golf is the exact opposite. There is virtually no athleticism required to be an elite golfer, it's almost 100% pure skill.
Every player fits somewhere on this graph, if you will. LeBron is so good because he has lots of both. Steph Curry (though athletic in comparison to your average Joe) is not a hyper athlete. He is, however, incredibly skilled. Rajon Rondo was a perfect example of more athleticism than skill.
Soccer is the same way. Every sport is the same way. Tom Brady is no more athletic than your average Joe. But he is IMMENSELY talented and very, very smart.