Are all children born with only either male or female body parts (other than exceptions), or is there actually a natural third gender with the physical characteristics of both genders, e.g. the child has the genitalia, or reproductive organs, of both sexes?
The concept of transgenders has nothing to do with sexual organs at birth. Transgenders is a concept that has to do with sociology, not biology. See the wiki link.
intersex (previously called Hermaphroditism) has to do with biology. The wiki link makes a great job at defining intersex.
In short: A transgender person is born anatomically either male or female (with eventually exception of intersex who go on to become transgender) and they then identify themselves to the opposite sex (or a third sex) rather than identifying themselves to their actual anatomic sex.
Note that this post is closely related.