I think it is really troublesome and kinda strange that we can see what is in the other side of a palisade, wall, etc.
There is a huge wall of bricks blocking my view and somehow i know that just a couple blocks away there is a huge farm of pumpkins?? But i thought something about that.
An area inside closed walls, palisades, brickwalls or even stones in the underground, should have a Fog of War protecting the information about the content inside. you could only see an unknown seetlement if you open a gate, break a wall of somehow enter there. Not only it is more realistic but it protects people from spying on your little house while you are sunbathing with your cows.