Office ran from 2005-2013. I was on a mission from 2004-2006 and then 2006-2007 I was a cadet at the Air Force Academy with limited TV access, which means I missed the first few seasons and never got caught up in the wave of excitement about it.
Same thing happened with Napoleon Dynamite - came home from my mission and my brother told me he had this great show I had to watch and I thought it was just the dumbest thing ever. Partly because it was the first video I saw in two years of being a missionary, partly because I just wasn't surrounded by the phenomenon and excitement of others enjoying it.
At least, that's my charitable explanation. It's also possible that they're just terrible, but there's enough people I know and respect who like both that I tend to base it mostly on timing and environment.