Answer, about 1-7 million dollars per mile. Because of this, right now improving road infrastructure is crippling many communities.
Biking infrastructure however is MUCH much cheaper. They are narrower, and bikes are MUCH less damaging than cars, so they last longer. It just FEELS more expensive because driving infrastructure costs are hidden and absorbed into the state/city budget as a need and alternative transit needs to be proposed as "extra". So it isn't necessarily more expensive to do biking infrastructure, it just feels that way because of choices we're making as society.