Colors won't peel, but will fade with time. print lines will be visible no matter how much clear coat you do. If you want to get rid of those you have to do some polishing. Downside is the colors are not very deep, between 1~2mm, so you might well polish off the colors if you aren't careful. Enough clear coats will make the surface smooth to the touch though, even without polishing.
I did a color sandstone print back in 2013, didn't clear coat it or anything. Colors lasted a just a few months sitting on my computer desk, not a bright room by most standards. Clear coat might have kept it longer but I can't say for sure. Haven't printed anything in color sandstone since so I don't have any pictures. Much more durable to print in WSF and hand paint it.
For little gift charms I'd do something like polished bronze steel, if you don't plan on doing dozens and dozens. Fairly affordable for small charms, around 3cm in the longest dimension, can be as low as 10~15$ per print. Infinitely more durable than sandstone, and won't feel as cheap. daily use will keep it polished.