Odds are... someone broke it. It doesn't take a huge amount of strength to break a timber house (64 STR to do any damage if wiki is accurate). Any decay during those 3 days would have decreased HP of the building, but as loftar pointed out, there is no probability that decay alone did it. I believe in astronomical random chances, but that's beyond astronomical random chance. The observable universe measured in angstroms isn't even that large of a number.
You may be too close to neighbors, or setting up in an area that they use to frequently forage or hunt. I'd suggest finding a nearby area if they can be reasoned with, or packing up and moving a good bit off if they can't be. Just as an example:
viewtopic.php?f=58&t=58400sMartins wrote:Watch over your head boys .....
You're more likely to get hit in the eye by a large piece of hail, thus blinding you, from staring up, or falling debris from an overhead airplane. I wouldn't suggest staring at the sky looking for such an event. You're more likely to go blind from staring at the sky than to get hit by even one meteor (Ok, I'd have to check the numbers on that one, but I think I make the point.)
Opinions expressed in this statement are the authors alone and in no way reflect on the game development values of the actual developers.