You need time away from playing to develop a real feel for your short game and your short wedge game. You also need time to actively train your swing with good feedback (instruction or at least videotape).

To get to a sub 5 handicap, you should expect to dedicate a minimum of 10 hours per week with at least 5-7 of that being practice time.

I was in that range in high school. Now, I am a 10 handicap because I play every other week or so. I just don't have the time. I keep telling my wife that eventually I will go back to golf, but I know that unless I dedicate some serious time, I will be about where I am: a decent golfer whose game is more valuable in a scramble than a real tournament because I don't have the consistency needed to really score well due to lack of practice.