It's all about wanting an agreement. Yes, citing eroding earthworks by walking on them or using the Devil's seat could be used to say no agreement can be drafted tobe clear and consistent but of course those things are accepted by common consent whereas climbing DD is not accepted by common consent so the "don't offend the feelings of others" clause nicely sorts things out.
If you know the vast majority of people would not want you to, don't do it. What could be clearer, more democratic or more workable than that?
Isn't that how we all live our lives anyway, or ought to?
This will sound a contradiction but Gladman and jj are the ones that have bottled the essence of what is needed. The only way we can protect undiscovered and out of the way monuments is to instill in the general public that something that can appear to be just a stone or a bump in a field may be important and needs to be treated as such, so if someone is messing around with it and you as a member of the public come across it then contact someone that will know and can investigate what the stone/bump is and lets all keep an eye on it.
If I see someone approaching a stone with a camera and a map, my alarm doesn't ring as it would do if it was a hammer a spade or a pot of paint.