Oaths being required to give the realm benefits goes against the (admittedly failed) concept of "neutral people in your area are still good". Now oathless villages are just people eating resources, there's no mechanical reason the realm cares that they continue existing. Now of course, this never really changed people's behaviours anyway. People who were farming RMT to pay rent still liked hermits, basement dwellers still liked killing them. Realms never made an impact (Vgates and barter stands definitely did though), but arguably it was an attempt (as sheep said, a skeleton to prop up a larger system possibly)
Nothing hermits/unaligned villages can do when it comes to realm v realm pvp. Punishing them for losing a region is arbitrary at best, actively discouraging involvement in realm at worst. All the villages that can field fighters will already be doing so, and as WB showed without some super abusable gimmick masses of relatively untrained and low stat militia don't do much in combat.
Unfortunately I don't think this suggestion would actually do what's intended to: Make more people care about and directly participating in realms (specifically, this suggestion leans towards pvp). PvP as it stands is "at capacity", all of the players who want to do it are already doing it. The only room to grow there would involve reworking pvp away from movement so that static objectives can be fought over. But movement is almost the entire sum of PvP skill expression and we've seen over the years that the Devs struggle to create engaging/fun pvp systems. Not to mention the likely significant amount of dev hours that would require.