Matters most for the Big 10, which actually does a lot via the Committee on Institutional Cooperation (Big 10 plus U of Chicago):
https://www.cic.net/home

Some thoughts from a Nebraska professor:
http://newsok.com/article/5436874

The ACC has the Atlantic Coast Conference Academic Consortium, which does some, but not as much: http://acciac.org/

The Pac 12 is trying to increase collaboration. The Big 12 has some small programs, though nothing like the Big 10:
http://wvutoday.wvu.edu/n/2015/05/18/big-12-fellowships-help-connect-schools-academically-application-deadline-nears
http://www.utexas.edu/ogs/faculty/ffp/

So it actually does matter, though it's obviously not critical to any individual school's academics.

And this regarding reputation:

In the study (link is external) in the July issue of the journal Sociology of Education, Stevens and two colleagues find that intercollegiate football leagues are composed of schools that tend to be similar in measures of academic reputation. What’s more, over time, scores assessing the academic reputation of schools admitted to any given league come closer and closer to the scores of those already in that league. “This reputation convergence seems to be independent of change in underlying academic quality,” said Stevens.

. . .

The researchers found that all of the widely publicized recent entries to the Big Ten, Pac-12 and Southeastern conferences, for example, are schools closely comparable in terms of US News reputation scores to the standing members. “And, of course, the most academically prestigious league in the country, the Ivy League, has offered membership to no additional school since its formation in 1956,” said Stevens. “That exclusivity is not based on sports alone.”

Moreover, Stevens and his colleagues discovered that the academic reputation scores of schools entering a given league move closer to their league average at a rate of 3 percent per year. “This may seem like a modest increment, but in the cutthroat competition for higher places in the academic pecking order, there are no small gains or losses,” said Stevens, a member of the faculty of Stanford Graduate School of Education.

https://ed.stanford.edu/news/ncaa-conference-affiliation-linked-academic-prestige-stanfordiowa-study

It doesn't seem unreasonable to expect that convergence can move from both directions -- let in a couple stinkers and the perception for the others starts sliding (and the Big 12 is already perceived as the worst conference in terms of academics). That and the opportunity for collaboration are why it matters.

Another example of potential collaboration: http://www.bigtencrc.org/member-institutions/

That research brings in funding and improves rankings and perceptions. I believe that some conferences also coordinate in lobbying for funding, which raises academic profiles and rankings.