Only about 20 universities operate with their athletics department in the black---BYU being one of them. The rest get subsidized from the university general budget.

Bigger issues are some of the others you mentioned. Students want brand new gym facilities, bussing to their buildings, etc. Then federal regulations require universities do X, Y, and Z, so then universities have to hire administrative staff to handle it. Here at BYU, we are hiring 2 full time, PHD administrators just to handle accreditation for education majors. Just for education accreditation! Why? Because the accreditation regulation are super onerous and can't be managed anymore without full time staff dedicated to it.

And now you have to have counseling services, disability services, multicultural services, service services, student services, faculty services, health services, mental services, emotional services, physical services, spiritual services, and on, and on. Universities are becoming little cities unto themselves. And that's what drives up the cost so much.