How much spare time would it need, I wonder?
That would probably depend on the monument. Maybe smaller ones could have been built by a handful of people, groups of hunter-gatherers as you say. Slowly moving stones a bit every time they pass, taking years and tears to complete the building. Adding bits, altering the plans that had been told to them by previous generations.