Well, it will much better if you could destroy things that was created by your own faster than anybody else.
For example if you own a millstone you should be able to damage it ignoring it soak value (better to ignore soak value of all building you build, not just roadsigns/millstones)