The word "Afrothere" comes from Latin Afer (referring to Africa) and Ancient Greek θηρίον (thēríon, meaning “beast”). In other words, "Afrothere" etymologically refers to animals with origins in Africa.
So really, Afrotheria and it's members are referred to based on their origins and not their physical characteristics. In other words, they needn't share morphological characteristics, but rather they all evolved from the same common ancestor from Africa.
From the Afrotheria Wikipedia page:
Afrotheria is a clade of mammals, the living members of which belong to groups that are either currently living in Africa or [are] of African origin...
They share few anatomical features but many are partly or entirely African in their distribution
These animals are all chordates of the class Mammalia, and therefore do share the common "basal" characteristics shared among all members of these higher taxanomic levels -- for example, a notochord (and backbone), hair, three middle ear bones, mammary glands, etc.