Even so that's like, 25,000-50,000 plant fibers? More reasonable would be 100 linen cloths to make, 5 to repair it. Would be cool if you could use a damaged balloon, but if you do you risk crashing amd taking mega damage.
Finally, you should be able to tell which direction the wind is blowing, so you know where the balloon will go.