Fair enough. I agree that those who put in the most work should get the advantage. It just sucks that cairn work is so boring for everyone involved and that after a few palisades a cairn is basically stuck in an impenetrable vault. 1 or 2 guys can out-manpower a large village because monitoring cairns is so boring that few people want to do it.
I think maybe cairn challenges should be able to bypass palisades, although that might just be a band-aid solution for the fact that as things are, breaking through any palisade is hell.