# How to calculate Fst from AMOVA

I calculated an AMOVA from a genind object, with one hierarchical factor. In the table I obtain there are SSD values (for my grouping factor,"Error" and total) and sigma2 values (for my grouping factor and "Error")

I have two questions:

• What does "error" stand for?
• How do I calculate Fst? Which values do I have to use?
• Welcome to Biology.SE. It may help if you could include the AMOVA outputs (or some of the outputs). Also, what software did you use to calculate your AMOVA? – Remi.b Jun 8 '17 at 14:41

I have never performed an AMOVA myself and I don't know what a genind object is. But ...

Error

Typically, "error" can also be called "residuals". In other words, it is the sum of squared deviation between the data and the regression line (just like in any regression model). You might want to learn a little bit about regression and ANOVA to understand your AMOVA.

Fst definition

From Nei 1973 (very good paper, I recommend reading it),

$$F_{ST} = \frac{H_S-H_T}{H_T}$$

, where $H_S$ is the average subpopulation variance in allele frequency and $H_T$ is the total variance in allele frequency.

What you probably got from the AMOVA output

You should have these info in your AMOVA output. If it only give you the total variance and the between group variance (which is what is used for the F-test), then just remember that the among variance is the total variance minus the between group variance.

• Thanks for your input! I found out that Fst cannot actually be calculated from the AMOVA, since I am not given the total variance in allele frequency. As to the error, in the AMOVA it refers to the variance which cannot be explained by the grouping factors I have fixed. A genind object is an object used by the library adegenet on R, which contains allelic information about a group of individuals. It enables you to carry out many types of analyses such as multivariate analyses. Sorry for not specifying it in my post! – Beatrice Baldi Jun 10 '17 at 9:23
• The AMOVA output probably gives you at least 2 of the following 3 statistics: Total variance, between group variance, within group variance. If you have two of them, you have the third one. Yes, exact you got it for the "error" term. As said, you might want to reinforce your understanding of simple ANOVA. Here and here are short videos that may help you. – Remi.b Jun 10 '17 at 14:40