In a small bowl, mix the warm milk with the yeast and a pinch of sugar. Let it sit for about 5-10 minutes until it becomes frothy.
In a large mixing bowl, combine the gluten-free flour, granulated sugar, salt, cinnamon, and nutmeg.
Add the pumpkin puree, melted butter, egg, vanilla extract, and the yeast mixture to the dry ingredients. Mix until a dough forms. If the dough is too sticky, add a little more flour, one tablespoon at a time.
Turn the dough out onto a floured surface and knead gently for a few minutes until smooth. Place the dough in a greased bowl, cover, and let it rise in a warm place for about 1 hour, or until it has doubled in size.