The 5 just wasn't for me. As my 17-year-old described it... "It's a car that somebody your age should be driving", and that hurt :). It's smooth, its a nice big(er) size, the seats seem a little softer. It's more refined... I felt old in it.
The 3 and 4 series will come down to personal choice.
I ended up getting a 4 series gran coupe. I really wanted a 2 door coupe since I'll be an 'empty-nester' in the next few months and i don't carpool anymore. However (this is going to sound stupid), My dog really likes to take rides in the car... and he would have a hard time getting into the back of a 2 door car. My boys are tall (6'8" and 6'5") and they can still 'mostly' fit in the back seat of my car. (but in truth... I got the 4 door for my dog)
In the end... the packages you want are: (3, 4, 5 series).
* M Sport package: This will give you a more 'sporty' look. The interior will have more M touches, the exterior will have blacked-out trim.
* Track Handling Package: This will give you tighter steering, suspension, and breaking.
Tick those two boxes and you'll have a great car (in any of the series).