Well, if we beat them, that 1 extra loss makes it much harder to be ranked
If we lose to them, they have a much better chance of being ranked. So of course we would see a lot more teams that we lost to on this list than ones we beat. Not to mention if they beat us they are probably really good