It isn't like a contusion or strain/sprain that's immediate, you can get hit and then the effects come on gradually over minutes after. I think they were overly cautious (rightfully so) in holding him out, but it's not surprising that things might've seemed fine initially but got worse over the next half-hour or so.