Jewish people make up a smaller percentage of the U.S. populace than the percentage of Muslims who were slaves, but no one really questions Jewish contributions to U.S. Society. To the degree that Borg believes Muslims haven't contributed to our country, its probably because they were enslaved and effectively forced to convert to the religion of their slave masters.
Put another way, one of the contributions of Christianity to our country is adopting a worldview that justified slavery, supported an economy that relied on slavery, and then forced the slaves to convert to Christianity so that any and all future contributions of former Muslims would actually be seen as contributions of Christianity.