The Ivy League plays 1AA football (I know, they are the Ivy League, and it's different), but look at other Top 50 schools with no football or 1AA football. Chicago U., Villanova, Georgetown, Pepperdine, Boston U., Northeastern...
It can go both ways. Do Top 50 schools Rice, Vanderbilt, Duke, Wake Forest, etc. benefit from having bad football programs? Would they take a hit in reputation if they were great in Olympic sports and competitive in basketball, but dropped football?
On the flip side, most of the top football schools are not real strong academically (undergrad).