Are a lot more pitchers than say PGs or QBs or even WRs or DBs. The longevity of some pitchers — 20+ or even 25+ Years makes it hard to compare such guys who were elite but never the best in a given year who were the best for a few years or even a decade. In other sports having the best player — especially basketball and especially with QB in FB — has a much greater influence on making or doing well in postseason. Some great pitchers never play in World Series or only are in the postseason a few times out of 20 years. Pitching is incredibly susceptible to injuries — now than, say, Being the best at slugging or batting average or even base stealing. And of course the nature of pitching — the precision and subtle effects of heat, humidity, lighting etc mean the performance can have a lot of variability — on and off days and the range of what that means. Just a few thoughts.
But the longer history of game, changes in ball/mound/equipment/style has a lot to do with it. There’s no good way to compare Cy Young or Mathewson with Bob Gibson or Clayton Kershaw.