Actually, jesters were usually considered smart and artistic, sometimes crazy, sometimes not. They were entertaining nobles, but were also allowed to mock and critizise them.
+PSY and +CHA/+INT would be more like it.
http://en.wikipedia.org/wiki/JesterVilliage idots on the other hand were indeed retards, it was probably caused by inbreeding in smaller villages. They were socially accepted. They didn't wear any special clothing, tho.