Update: I was able to get to lvl 4 by just moving the pyre a few tiles, with the same amount of str. I was finally able to build that windmill. I find that to be a must for ease of play, even as a hermit.
Now I am on level 5, and working to get to lvl 6. I will not be able to go deeper than that, because of the requirement for that rock crystal. I have yet to see one. I have had bad luck with the types and amount of ore I get alas. Sadly, you do not know that before you start to make mine holes. Or at least I am not able to. No precious metal:(
I made a village and claimed the space down with village banners instead of using personal claims. Eveything is of course palisaded and locked. Building a mine hole in the basement of a house is a neat possibility I will do in later worlds, but then with the house palisaded as well with a visitor gate to the main base. I can not see there is any other options if security is a thing.
I had to rebuild the visitor gates from the base to the mines once, they faced the wrong way.
Now I am contemplating building a stone tower as a signature building.