Maybe were not so mad when we are mooching in the countryside and speculate something similar when we see impressive hills, mounds or hummocks that have later been used to construct castles/churches etc...
Stands to reason that if our forbears found a large 'hill' somewhere strategically/culturally important for them, they are gonna re-use it. After all, if you don't have to build a massive base to start with, your half way to wherever you need to be.