Imagine there are two populations X and Y, and for each population you have the genotypes of each individual in that population (e.g. Aa, AA, aa, etc.), but for multiple loci (e.g. AABb).

How could one calculate the gene flow between population X and population Y?

An answer would ideally recommend either or:

  • An R/Python/C++ library, or any other software tool
  • the breakdown of the maths, for which I would be much obliged, but not entirely necessary in the answer

Background reading

Related questions

  • $\begingroup$ @Remi.b here's the question in case you are interested in finding out the answer (great hearing from you too by the way) $\endgroup$ Aug 19, 2015 at 17:33

1 Answer 1


Theoretical Background

From Slatkin 1991, at equilibrium

$$F_{ST} = \frac{1}{1+4Nm\left( \frac {d}{d-1} \right) ^2}$$

, where $N$ is the per island population size, $m$ is the migration rate and $d$ is the number of islands ($d$ stands for "deme"). As $d \rightarrow \infty$, $\left( \frac {d}{d-1} \right) ^2 \rightarrow 1$ and Slatkin equation becomes the standard Wright's equation $F_{ST} = \frac{1}{1+4Nm}$.

From the above, it is easy to show that

$$m = -\frac{(d-1)^2(F_{ST}-1)}{4 d^2 F_{ST} N}$$

For 2 islands, it becomes

$$m = -\frac{9}{16} \frac{(F_{ST}-1)}{F_{ST} N}$$

In consequence, in a 2 island model the expected $F_{ST}$ is almost twice as low $\left(\frac{9}{16} = 0.5625\right)$ than in the infinite island model.

The above equation is an approximation that assumes...

  • low mutation rate (would not apply to microsatellite data)
  • relatively low migration rate (there is a $m^2$ term that is dropped)
  • symmetric migration rate (backward and forward migration rates are equal)
  • equal population size per island
  • equilibrium!

Further Reading

I recommend reading this paper (Slatkin 1991), it is IMO one of the best on the study of $F_{ST}$. I would also recommend Nei 1973, Slatkin 1985 and Whitlock and McCauley 1998.

The specific software to use depends on the kind of data you have. In any case, if you average $F_{ST}$ over a number of loci, make sure to use Weir and Cockerham 1984 method.


The first step for you is to calculate your $F_{ST}$. Assuming you have a number of SNPs, you will want to use Weir-Cockerham estimate of $F_{ST}$. There are a number of solutions to compute such estimates. One solution to calculate this is vcftools. The following (Bash) will do the job

./vcftools --vcf MyFile.vcf --weir-fst-pop individual_list_1.txt
--weir-fst-pop individual_list_2.txt --fst-window-size 800 --fst-window-step 100

, where MyFile.vcf are your data in vcf format (you might eventually want to use PGDspider to reformat your data). individual_list_1.txt and individual_list_2.txt are files containing the list of individual names (seperated by \n) belonging to the first and second population, respectively. The options --fst-window-size and --fst-window-step allows you to compute the estimate over a sliding window. I chose arbitrary numbers above. EggLib is another good alternative.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .