Actually, we not only consider that all human beings belong to the same species (Homo sapiens) but even that we belong to the same subspecies (Homo sapiens sapiens). So, does it really makes sense?
Concept of species
First, please note that the concept of species is more arbitrary than the most layman would think. I wrote my opinion about the concept of species in this post.
Concept of species in sexually reproducing lineages
In sexual lineages, the existence of different species is generally defined by the existence of a reproductive barrier (see reproductive isolation and see UnderstandingEvolution > biological species concept).
A reproductive barrier can be caused by either pre-zygotic or post-zygotic isolation. I will mainly disregard potential case of pre-zygotic reproductive isolation in this answer because such isolation depends highly on the culture and not only on human genetics. I do not mean that pre-zygotic isolation does not matter or are irrelevant, I just decide to focus on post-zygotic isolation because, as being very independent of the culture, it makes its study much easier.
Reproductive isolation might be partial typically leading to cases of outbreeding depression (when it comes to post/zygotic isolation).
If we had many documented cases of outbreeding depression, it would suggest that we should reconsider the decision of considering all human being. So the question is really: Do we have documented cases of outbreeding depression?
Do we have documented cases of outbreeding depression?
Many cases of inbreeding depression have been documented in humans (McQuillan et al. 2012, Strauss et al. 2013, Lettic et al. 2008, Gellera et al. 1990) but cases of outbreeding depression seem much rarer if any!
I spent some time screening through the literature searching for potential evidence of outbreeding depression. The only paper I found is Udry et al. 2003.
Udry et al. 2003:
First, note that they do not place their results in the context of our present discussion and do not talk about outbreeding depression.
They report that children of "mixed-race" in US colleges report having more behavioral troubles. One would obviously note that there could well be true that "mixed-race" children experience a different environment (incl. social environment) than "non-mixed-race" children. Such results, therefore, does not suggest the existence of any outbreeding depression.
They also report more skin problem in mixed-race children. Unfortunately, 1) their p.value is only slightly significant and 2) they track tens of variables without correcting for multiple comparisons and only one came out significant causing the expected family-wise false positive rate to be higher than the observed family-wise positive rates (see false positive and related concept in statistics).
So, in short: No, there is little to no evidence of outbreeding depression in humans
Potential publication bias and taboo
It is not impossible that there is a publication bias in which researchers looked for inbreeding depression more than for outbreeding depression. One would note also that it would probably be quite taboo to attempt to suggest that there are several species of humans for social reasons and such taboo could cause a publication bias too.
I personally doubt it would be the case that such publication bias exists. Note also that if the evidence of outbreeding depression were obvious and common, then we would not have a problem telling that there are several species of humans. In essence, if there is a publication bias that is causing us to fail to see existing outbreeding depression, it must mean that existing outbreeding depression would be relatively rare and with small effect anyway.
Other definitions of species and taboo
Well, the issue is that there is no other commonly accepted definition of species. Typically, above I only considered outbreeding depression and I did not consider a potential pre-zygotic barrier that would result from geographic isolation or from cultural reasons (e.g. maybe on the ethnic group does not like the appearance of people of another ethnic group and vice versa). I don't doubt that for social reasons, we tend to be quite happy that our only definition of species support the idea that all humans belong to the same species.
It is hard (for me at least) to tell whether we might tend to be more permissive in other lineages and naming two sister lineages as belonging to different species.
Why same subspecies and not only species?
Under the biological species concept, H. sapiens and H. neanderthalis should belong to the same species. This is why we now rather call them H. sapiens sapiens and H. sapien neanderthalis. I suppose it would feel painful to many to not highlight our differences with H. neanderthalis and so, we decided to discriminate.
Again, it is hard (for me at least) to appreciate how this subspecies discrimination in the Homo lineage compares to discrimination among potential subspecies in other lineages.