Goose Bumps (a.k.a. - Goose Pimples) are caused by the Arrector Pili muscles in your skin:
Their purpose is to increase the distance from epidermis to the top of the follicle, and increase the area between follicles. This has the effect of increasing the amount of air trapped between the hair and surface of the skin. In other mammals the end-result is an increase in the insulatory properties of their fur, meaning they will stay warmer or cooler depending on their current needs.
Another function in our mammalian cousins is to simply appear larger, and thus more of a threat to any predators or dangerous animals. Much like the shoulder and lower back hair on this dog:
Thermoregulation in humans has evolved since our furrier ancestors; we sweat to cool off, and shiver to keep warm. We also use tools and weapons, can create fire, and have other ways to communicate our lethal capabilities.
Goose Bumps are the human body utilizing old instinctual reflexes in the proper situations, but we lack the thick mattes of hair our ancestors had for the reflex to have any significant effect. They're a fun Easter-egg, a leftover that doesn't do us any harm and are an easy way to tell if someone is cold or scared - which might be useful under the proper circumstances.
