If the purpose of "goosebumps" is to keep us warm, why is it so ineffective? Or are the changes subtle, but important?
Cutis anserina (goosebumps) are caused when the sympathetic nervous system induces the arrectores pilorum muscles at the base of each hair cell to contract and raise up the hair. Zoidberg has posted an answer describing the mechanism as I was writing mine, so I'll move directly onto effectiveness.
The reason that, as you say, the mechanism is not particularly effective in humans is our relatively small amount of body hair. As mentioned the purpose of the erector muscles causing the hairs to stand up is to allow for air to be trapped at the surface of the skin rather than absorbing heat from the body then taking it quickly away. Mammals with denser fur/hair are likely to have a much better insulating effect from goosebumps.
Some animals have adapted to use the mechanism for another purpose, however; the porcupine being a prime example. When threatened, the porcupine uses its arrectores pilorum muscles to elevate all its quills, making it appear larger and more threatening or to actively repel an attacker. The extra rigidity that they gain from doing this1 also helps to contribute towards making their spines a very effective defence:
Goosebumps have also been reported in humans during periods of stress, anxiety and even sexual arousal. This is related to the stimulatory effect that adrenaline has on the sympathetic nervous system which, as mentioned above, is responsible for the stimulation of the erector muscles.
When your brain, the hypothalamic temperature centers in particular, detects that the temperature is too warm or cold, it initiates a number of controls to try and correct this.
Goosebumps appear due to piloerection. This is one of the reactions that occur when the temperature is too low.
This causes hairs to stand on end as a result of contractions in muscles attached to hair follicles called arrector pili.
This particular reflex is not actually important in human beings. However, in animals, this mechanism allows entrapment of a layer of air allowing insulation. This way, the heat loss is greatly reduced.
The other mechanisms are very adept at maintaining temperature in the human body. These include sweating, dilation (vasodilation) and constriction (vasoconstriction) of skin blood vessels and increasing and decreasing the body's heat production.
For example, when it's too hot, dilation of the skin blood vessels can increase heat transfer by up to eight times.
Source: Guyton and Hall. Medical Physiology. 11th ed. Elsevier Saunders.