As long as we only look at humans the differences are clear: males have chromosomes XY, produce sperm and don't get pregnant. Females have chromosomes XX, produce egg cells and bear babies. But when you consider other species, things are more complicated: in birds, it's females who carry the Y chromosomes. In pipefish and sea horses, it's the males who get pregnant. So maybe the only reliable criterion to tell if an animal is a male or a female is looking at its reproductive cells, and decide if they look like spermatozoa or egg cells. But then there are male and female plants, where none of the methods above applies.
So how do you tell, in general, who's male and who's female in a species? Or is the distinction arbitrary?