GWAS: why is replication in another cohort so crucial?

Question

Almost all the landmark GWAS (Genome-Wide Association Studies) reviews agree that, for a GWAS finding to be valid, it needs to be replicated in an independent cohort. What exactly is the rationale behind this? Is this criterion still valid even when the objective is looking for genes to do functional studies, instead of a clinical perspective of searching for susceptibility loci? Considering the difficulty in replicating some phenotypical characterizations in a large numbers of subjects, for a biologist this whole thing seems irrelevant.

Consider the following scenario: Suppose I have 5000 phenotyped individuals with a budget to genome-wide genotype them all. The phenotyping requires state-of-the art methodology with immense costs, and the cohort was phenotyped through another grant. What would be the point of dividing my cohort to two as a discovery and replication groups, other than to save money in exchange for stat power. To this, add the subsequent functional studies on the associated loci, what would the point in genotyping a few SNPs in 500 more people be? If one can indeed find that additional cohort, wouldn't it be a better option to combine any available cohorts in a meta-analysis, rather than using them for replication?

Answer 1

As an elaboration of my comment.

Summary: Replication is required in GWAS studies to account for non-random technical biases.

An example of such bias is, for example, a chip used for genotyping giving consistently incorrect genotypes for a locus. In this situation adding more subjects will not correct for this effect and therefore the only solution is to genotype additional subjects with another method (e.g. another chip or experimental protocol). Another source of non-random technical bias is at the phenotype measurement level. You might want, if possible, to measure the phenotype using other techniques to make sure that bias will be corrected.

A replication cohort does not need to be huge and the actual size needed will largely depend on the effect size of the SNP seen in the discovery cohort. A "power-to-detect" calculation would help in predicting the cohort size needed. For a SNP to be significant, it must pass Bonferroni correction but as you can run a statistical test at the candidate site only the correction will be usually not very astringent (i.e. you might run association tests on "only" a few dozens or hundreds SNPs).

Historically GWAS replication was also required to correct for population structure but as the tools evolved substantially (e.g. the use of principal component analysis) and the cohort sizes are now substantially bigger compared to the first GWA-studies, this is less a concern. What was required is to sample independently other subjects from either the same or a different population.

For your situation what you could do is to split your cohort into a discovery and a replication panel, genotype your subject in the replication panel using an independent technique, eventually do the same with the phenotyping and replicate you candidate SNPs. The replication panel needed is usually must smaller than the discovery cohort for the reasons stated in the previous paragraph.

For your last question about the meta-analysis. Yes this could also be a way to go but be careful that you will still need a replication cohort to validate SNPs found this way therefore resulting in the exact same problematic you described.

I hope this help!

Answer 2

As with all serious scientific result GWAS results need to validated by others. In this case it think is extremely important because these studies link mutations to diseases or in more general given genotypes to phenotypes, thus pointing out possible causes. So validating these results with the use of independent "samples" is indeed crucial. But as I said all serious (noteworthy) scientific finding should be reproducible.

Edit: the point in it is randomness and sub sampling. Such large cohort should not come from a small sub-population but many regions of the world, and by randomly dividing your individuals to two or even more groups you ensure that other background effects such as local gene allele variation frequencies, life-style differences etc average out. Imagine a scenario where you have 90 individuals in 60 of them you can link the genotype to the phenotype. that is 2/3 of the individuals. But if you take 3 sub samples of 30 individuals and get 7/23 , 17/13 and 6/24 (not linked / linked), that is 76.6% (23/30) , 43.3% (13/30) and 80% (24/30). From these you can get tbe same average of 66.6% but with standard deviation (15,5), and confidence interval not just a number.