oh I missed out that you said "Merely setting foot on Silbury damages it in a way that setting foot inside a house at Skara Brae wouldn't damage that structure"
I'm not sure there's any difference, you can say that one person climbing silbury - I climbing silbury - does not leave any noticeable damage at all. Likewise at Skara Brae. But the important thing is the cumulative damage to our irreplaceable prehistoric structures.
That's why people aren't allowed to wander through Skara Brae, right? What other reason would there be? It got nearly 70 thousand tourists this year!