Doesn't seem like you're able to dig on thicket so I suspect that's likely why the dirt simply decays. I assume you would have to modify the thicket terrain around the ledge first before being able to do anything about it, though it seems like you did that as much as you can already so if the remaining tiles that are too close are preventing its removal I'm not sure there's much you can do about that.
When discussing ledge removal in another thread this was also suggested, I've always just dropped the dirt myself but if this method works it may save you some time. Provided you can change enough of the thicket around the ledge to get rid of it.
Vesena wrote:IIRC the easiest way to remove a cliff was to ctrl-shift-right-click with soil on the tile adjacent to its base.
Try raising the elevation on any tiles nearest to the ledge that are not thicket and see if that's enough to eventually get rid of it.