# Is $F_{ST}$ a probability and a correlation coefficient?

$F_{ST}$ is one of the most famous and most important statistics of all of evolutionary biology. Yet, many people misunderstand it or misuse the classical results from the literature on $F_{ST}$ (Whitlock and McCaughley,1999).

Fixation Index in infinite islands model

In a panmictic population, the probability of identity by descent $F(t)$ in generation $t$ is the probability of sampling the same allele twice plus the probability of not sampling the same allele multiplied by the probability of identity by descent in the previous generation

$$F(t) = \frac{1}{2N} + \left(1-\frac{1}{2N}\right) F(t-1)$$

, where $N$ is the population size. Here I am assuming diploid population and no mutation. In an infinite allele model, these probabilities must be weighted by the probability that none of the parent migrated in the previous generation.

$$F(t) = (1-m)^2\left(\frac{1}{2N} + \left(1-\frac{1}{2N}\right) F(t-1)\right)$$

, where $m$ is the migration rate between any two demes. Setting $F(t) = F(t-1) = \hat F = F_{ST}$, assuming that $m$ is low and solving for $F_{ST}$ yield to the classic result from Sewall Wright

$$F_{ST} = \frac{1}{1+4Nm}$$

Definitions of $F_{ST}$

$F_{ST}$ was defined by S. Wright as

$$F_{ST} = \frac{var(p)}{\bar p(1-\bar p)}$$

, where $var(p)$ is the variance of allele frequency among population and $\bar p$ is the overall average allele frequency.

From Nei (1973)

Wright showed that the variation in gene frequency among subpopulations may be analyzed by the fixation indices or F-statistics. He derived the formula $$1 - F_{IT}= (1 - F_{IS}) (l-F_{ST})$$ where FIT and F1s are the correlations between two uniting gametes to produce the individuals relative to the total population and relative to the subpopulations, respectively, while $F_{ST}$ is the correlation between two gametes drawn at random from each subpopulation. $F_{IT}$ and $F_{IS}$ may become negative, but $F_{ST}$ is nonnegative.

Question

In the derivation of the fixation index in infinite island model, $F_{ST}$ is a probability of identity. In the quotation from Nei (1973) (and other sources), $F_{ST}$ is presented as a correlation coefficient. In Wright's and Nei's definition, I don't see the relation from the equations to either a probability or a correlation coefficient. Can you help to clarify this for me?

For example, I would expect that $E\left[\frac{var(p)}{\bar p (1-\bar p)}\right] ≈ \frac{1}{4Nm+1}$, where $E[X]$ is the expected value of the variable $X$. Can you demonstrate that this is true?