I've said before that it's my firm belief that nobody should climb the stones. Or other type of site, of course. Because although one person probably won't cause any damage, if every single visitor had that mentality, and every single visitor believed they had the right to climb the stones/whatever, as they, personally won't cause any damage, then you're looking at potentially hundreds, thousands, our even millions in some cases, of people "not causing any damage"! And you know what? Over the years, that amount of boots on rocks WILL cause damage! Just look at the steps in old castles, churches, even town halls! Some sites are so reconstructed to the point that it probably doesn't matter, but it's best not to take the risk, eh? I know it'll upset some, but I think it's a selfish act. Nothing personal - I'm referring to EVERYONE who thinks it's ok.
S'obvious, really. Just my opinion, of course.
I think you've put your finger on it, we all think we aren't doing any harm but multiply that by thousands ... definitely applies to people who insist on climbing Silbury. The same argument applies to all the things we do to damage the environment - one unessential car journey doesn't do any harm but multiplied by a million other journeys we are poisoning the atmosphere (ditto plane journeys). I personally would never climb on a stone monument and will think again before walking onto a long/round barrow - which I have done to take photographs in the past.