If the easement is recorded, there isn't much he can do about you using it for access, but if it is an implied easement due to necessity, I would approach him and let him know you are developing the property so the two of you should formalize it to avoid any future legal confusion over the easement.