Like many who don't follow basketball closely, you may correlate arena size with the quality of the program.
You might be surprised to learn that Villanova seats 6500, Gonzaga 6000, and Duke 9000. Etc.
Portland isn't great, but it has nothing to do with arena size.