Trees are big and tall. Most of the fruits do not hang low.
Hearthlings are quite short.
I think that you should be able to collect like only 10% of the fruits of a tree without ladders, and the rest 90% when using ladders.
Ladders would be a buildable object that you could lift and carry them around, then you would right click a tree while carrying the ladders, and then "assemble" the ladders below the tree.
The current harvest from fruit trees may not need to be lowered, but instead, you would get even more fruits with these ladders.
Something like in the images above.