With some careful cuts, a bit of paint, and some foil tape, I realised I could very quickly make a cotton bud into a small pencil.
This success prompted me to try even smaller, with the broken bristle of a hairbrush and the point from a cocktail stick.
But the correct scale for my puppets is a little bigger, so the last pencil I made from a lollypop stick.