Looks like it's not a simple question.
According to wiki, we have 2 softcaps. First one from skills - 5th root from dex*perc*psyche*masonry*lore. Second one - from quality of gemcutter's wheel direct and the second one softcapped first one. The result of second one softcap is the chance of succed to cut the gem.
So. What if result of rot will be something like 300 with quality of gemcutter's wheel will be 100? In theory, chance to succeed will be 200%. What will be with gem with this chance? How does this work? And how do i suppose to find quality of my gemcutter's wheel, if inspect gives only what size of gems i can cut?