# Shannon diversity index: Pi relates to all found species, or number of species by site?

I am trying to dig into Shannon's species diversity index to compare the species diversity between different sites. For this, I need to calculate the 'relative abundance of species within the community'[details here]. What I do not understand is if this relative abundance relates to all species in a sample, or all found species?

In my example, I have in site1 6 species: a, b, c, d, e, f and in site2 two species: g, h. Should I calculate the proportion by species by site relative to 6 and 2, or to 8 (6+2, as species do not overlap between the sites? This is dummy example, likely species will reoccur).

• If calculated by total number of species (8): Shannon H = 1.11 and 0.820,
• if calculated by total number of species by site (6 & 2): Shannon H = 1.29 and 1.26.

I wonder which one is correct? The general trend is the same (site1 is more diverse then site2) but differences between indices seems quite large.

Dummy example to calculate Shannon's H step by step with dplyr. The commented out mutate allows to change if distinct number of species and their sum will by calculated by site or by total.

# Create dataframe with two sites and 6 and 2 different species. Area indicate abundance of species.
d<- data.frame(site = c(rep("s1",6),  rep("s2", 4)),
species = c(letters[1:6], letters[5:8]),
area = c(59,12,11,10,5,3,
10,5,21,20))

# Calculate sum of the area
# calculate distinct number of species, vary if the distinct area calculates by sum or by total??
d %>%
mutate(species_n   = n_distinct(species)) %>%   # !! should 'mutate' by placed before the grouping variable or after?
mutate(species_sum = sum(area)) %>%
group_by(site) %>%
#mutate(species_n   = n_distinct(species)) %>%
#mutate(species_sum = sum(area)) %>%
mutate(spec_pi     = area/species_sum) %>%
mutate(pi_ln       = abs(log(spec_pi))) %>%
mutate(sp_shannon  = spec_pi*pi_ln ) %>%
mutate(shannon_comun = sum(sp_shannon))