# Measure of diversity, accounting for known species

I'm interested in measuring the diversity of each sample out of many independent samples. I know a priori all possible species that could appear in a sample, and expect the counts of species to be from 0 to about 10, although often times there will be none of a particular species within a sample. For example, some data might look like:

Sample # | Species A | Species B | Species C
--------------------------------------------------
1          0           4           0
2          1           1           1
3          0           1           1
4          2           2           2
5          0           1           0


The most diverse sample here is #4. The least is #5. Theoretically, #5 would also be the least diverse ever encountered (assuming there is at least one of some species in each sample).

I encountered trouble when using Simpson's Index of Diversity since that formula would make the diversities of Sample 2 and Sample 3 both 1.0. Intuitively, with this data, I am confident that #2 is more diverse. This is because I know all possible species that may show up.

I'm relying on this definition: $$D = \frac{\sum n(n-1)}{N(N-1)}$$

Are there measures of diversity which account for a priori knowledge of possible species (and perhaps their bounds) and expect near-zero counts?

• Without genetic data, the standard other measures of biodiversity are Shannon-Wiener index and, simply, species richness (aka. number of species). I'm afraid that both Shanon-Wiener and species richness would rank sample 2 and sample 4 equally. Commented Jan 10, 2019 at 9:03
• There isn't a single "true" measure of diversity, and different aspects of diversity can be taken into account (abundance, evenness, uniqueness etc). Different indices also weight these aspects differently. So you need to carefully define what you are after before choosing an index. Commented Jan 23, 2019 at 13:00
• Also, I question your calculation of D; to me the value for 2 & 4 would be 0.333 and for nr3 0.5 (i.e. 1-D equal to 0.667 and 0.5, respectively). Commented Jan 23, 2019 at 13:04
• @fileunderwater Thanks for the comment. I realize there isn't a single "right" solution, I'm just looking for the most suitable: something that takes into account the number of unique species in addition to the overall abundance of each (perhaps with knowledge of possible species). Pardon me for not being from the biology community (this is actually a linguistics problem, but I'm reaching out to biology because there may be a fitting link) Commented Jan 23, 2019 at 19:49
• @AlexL No worries about being from another field, especially since the diversity indices have quite wide appeal. I'm currently working a lot on scientometric data, and diversity indices (often borrowed from ecology) are common there as well. I think it would be useful for you to look at Hill-numbers, which basically are ways to rescale diversity indices (measures of entropy) into effective numbers of species. This makes it easier to compare numbers from communities/sites with differnt levels of species richness. Commented Jan 24, 2019 at 15:15