Hemp is basically the red stepchild of clothing material because it has the same attrib as linen, but also takes 6 fibers to make cloth out of. I don't mind needing 20% more space to make the same amount of clothes if it gives a difference like the linen/wool/nettle options do for low level production of clothing.
What attrib to switch to? I dunno.. Maybe carpentry? Its a much more durable material than either of the three other options.