Given that you can drop a minehole anywhere and then wall it up--better to build the wall first, imo--I'd either find the best place near by with minerals to drop it or put it inside your village with a bit of extra security (read brick wall it as soon as you can, maybe even double brick wall). You can reasonably dig to any minerals close by without supports if you are very careful.
Just my opinion, though. Anything you do, plan with security in mind. Raiders can dig into any minehole or cave you have inside your village walls.