Tell him that now that there's been an appraisal done that it could be accessed by future borrowers on the property and the same issue will arise (HIGHLY unlikely, but not an untruth). He might as well deal with you, since you're already down the track a ways to a purchase and at least you're willing to meet him in the middle, the next person to see that appraisal may not be.... No sense going back on the market only to have the same problem again but have lost months of market time, mortgage payment, utility cost, upkeep, property tax, etc.

Pretty easy to make your case. I'd have a check in your hand written for $2500 to the title company to wave in front of him, show him you're ready to make the deal.