Yeah, that's been my understanding, MadNomad. For now, I've just been trying to mark the gems I do find so just maybe I can get lucky and make gem hunter a little less painful.
Opinions expressed in this statement are the authors alone and in no way reflect on the game development values of the actual developers.
I just finished gemhunter on 2 chars... and it wasnt bad (comparing to fisherman). I got like 2 quests to mine specific gem/cut, and I skipped those. Atm I have 2 cupboards of gems - mostly small, bit also few grand/jotun. You just need to mine in many places.