I do not see why should scary situations make our skin "crawl". In scary situations, we supply less blood to our skin (which makes it paler), so shouldn't its activities reduce? Goosebumps should take up some energy to raise the hairs. Shouldn't that energy be conserved?
Goose bumps are based on a pilomotor reflex where tiny muscles at the base of hair follicles contract, which leads to the "bumps" and also raises the hair in the follicle.
It's often explained as a vestigal reflex, dating back when our ancestors had more hair than the modern human. The "puffing up" of fur has two effects, which today still both can be observed:
- Raised hair traps air in the fur, thus minimizing heat loss in the cold.
- Raised hair (often combined with certain postures) makes an animal appear larger, thus intimidating an opponent or predator. This is the scenario you mention in your question.