Natural selection "selects" the trait that is less advantageous and removes/ reduces its frequency. It is like a filter paper. All solids are filtered out of mixture while clear liquid is collected. So if there is a difference in fitness of organisms by variations in traits, then only they are SELECTIVELY removed. Now, if a person without an earlobe would have had an increase in fitness, it would have had eliminated over time. And it might (I said might) be that there IS some pressure but it might be so slow that we can not realise it. But in case of earlobes, evolution will be evident in the case of elephants and Jackrabbits.