That's because you're talking about the top 12-16 teams in the country.
You'd expect a team ranked in the top 16 in the nation would be a daunting task. Only way it's easier is either if you have a top 4 seed or if the team you'd be opposite gets upset in the 1st round.