It's currently most optimal to wear armor every time you go outside. It's also currently most optimal to take off your armor every time you fight an animal so it doesn't break your armor, because all armors have really low durability.
I'm ok with the durability per se, it's very possible to shatter an opponent's defense and that's a good thing. But there should be a small grace area where if im only taking 5 damage at a time from animals im fighting it just shouldn't affect my armor.
It's not like this prevents you being able to defeat strong opponents by chipping them down since that chip damage is coming from armor pen anyway. All it does is mean that you can wear armor outside and during PvE so long as you either keep your openings low or just be strong enough you can't be opened.
At the very least this should be a thing for large, expensive armors like bronze plate.