College football is weird, and good teams lose to not so good teams all the time. USC is immensely more talented than we are and the coaching of both teams is on about the same level. Play that game 10 times and I'd be willing to bet USC wins 8 of 10.
When we get to the end of the season, we'll probably look at the body of work as a whole and see that we had no business beating either USC or Tennessee.