There is plenty of fun that comes in the starter pack. That alone is a great game that we stuck with for about half a year without any extras, and we were still having fun within the world building/toy box stuff.
The funny thing was, our kids wanted a $15 character add-on more than they wanted a different game for $??. It was simple to sprinkle in a few for bdays and the following Christmas.
So, "infinity" may certainly mean the number of extra characters they want you to spend money on, you can keep things pretty basic and still have a lot of fun.