I found clay pretty easily. I still haven't exhausted the spot despite grinding clay.
EDIT:
Ok foot in mouth it ran out right after I posted. I hopped in my boat and started jumping around digging for clay. It seems to be laid out as small scattered nodes. One spot would run out, and different tiles next to it might yield 4 or 5. 30 tiles down or so and another spot.
I'm curious about whether they respawn in the same tile, so I've been marking nodes so I can verify this.