We have some in one of our grazing areas, can't say I've ever seen them at it.
It grows all over West Somerset, which is a big sheep area, and no-one seems to think it does any harm.
Maybe best not to plant only elder, but provided it forms a part of a mixed hedgerow, I'd have thought you'd be fine. And it makes beautiful champagne, cordial and wine