How about the berries of foreign plants? I often see strange bouquets with weird berry-bearing plants.
But then would you also cut down non-native trees? Horse chestnuts, for example?
That's probably going a bit too far doncha think. It's daft to try and change what's happened in the past, but that doesn't mean you should perpetuate the problem in the future.