How one drives dramatically alters the safety of driving, so comparisons of averages between the two really aren't that great. Many car fatalities are not random events, but are caused by driver errors which can and are avoided by safer driving. Or, how about the idea that it's not fair to include deaths on city streets because such short trips are not possible for airplanes?
I suspect that in many/most ways air travel is safer, but the actual analysis is not that simple.
See this abstract (it is older, but poses some interesting questions about the difficulty of truly equal risk analysis):