How do I cancel build signs?

[estan]

A scrub village has decided that I'm too close to their "Dew" trail and warned me not to get in their way. Of course, they did this after I started building a house where I didn't know a trail existed. Anyone know how to cancel that?

[qoonpooka]

You can destroy the sign with a single swing. Adventure->Destroy then click the sign. You lose all materials you've built into it.