Same for me, too. In my case, it now happens almost instantly, and lasts for a day, sometimes two, even if the other symptoms don't appear. My skin feels warm, as if blood was rushing to the skin all over my body, but I feel cold inside and start shivering at random times even if the environment is warm. Sometimes it's more extreme, sometimes less, but it happens every time, and now that my symptoms have worsened, it happens with arousal too.
Exercising for a bit used to make this symptom disappear, but now that I have exercise intolerance I can't do that. Sunlight seems to help, though.