If you are at the equator and start moving north, the further you travel, the lighter the skin of the indigenous peoples. Considering that we live on a ball, why do we not find the same traveling south from the equator? This relationship does not hold for individuals as you move south from the equator. At the very least, one could say there are no blond hair blue eyed peoples south of the equator.
I rather would say that the lack of North/South Symmetry in pigmentation is that we forget how quickly human beings have spread. In prehistory, people have come to populate every continent over perhaps the last 60,000 years.
While in that time its clear that several mutations have popped up to influence skin color, they are pretty rare compared to the speed with which we migrated over the globe. The dark color of our earliest ancestors has been lost and regained more than once, but not so often that the selection pressure of latitude have made indigenous in the tropics of south america or south east asia uniformly as dark as africans. Or for that matter have lightened the skin of pre-colonial South Africans.
One might guess that clothing and shelter have reduced the selection pressure on pigmentation further over the past few thousand years.
Individual cases should be studied - this is just a sketch - but the general thought here is that the pigmentation of human beings have more to do with where they have been and how quickly they have recently come from in their migrations.
Summarizing the comments below: There is a correlation between pigmentation and latitude. The answer is really a combination of latitude, how long a people has been there, and the speed at which pigmentation mutants show up. It doesn't change so quickly that all people are equally dark at the same latitude or so slowly that you find completely pale people at the equator.