If they aren't supposed to have it - the authorities will make them take it out.

Now - I'll admit that some plants are more annoying than others. Having a neighbor with a tree that hangs over and drops all kinds of crap is more annoying as it now starts to actually effect me and my yard. But it's still their tree and their deal.