I would take a copy of the invoice over to him and them them know what half of
the fence costs, acknowledging they said they couldn't afford it now but maybe in the future... If they have a conscience then they will pony up for their half. If not, it probably won't change your relationship very much.