Almost all the landmark GWAS (Genome-Wide Association Studies) reviews agree that, for a GWAS finding to be valid, it needs to be replicated in an independent cohort. What exactly is the rationale behind this? Is this criterion still valid even when the objective is looking for genes to do functional studies, instead of a clinical perspective of searching for susceptibility loci? Considering the difficulty in replicating some phenotypical characterizations in a large numbers of subjects, for a biologist this whole thing seems irrelevant.
Consider the following scenario: Suppose I have 5000 phenotyped individuals with a budget to genome-wide genotype them all. The phenotyping requires state-of-the art methodology with immense costs, and the cohort was phenotyped through another grant. What would be the point of dividing my cohort to two as a discovery and replication groups, other than to save money in exchange for stat power. To this, add the subsequent functional studies on the associated loci, what would the point in genotyping a few SNPs in 500 more people be? If one can indeed find that additional cohort, wouldn't it be a better option to combine any available cohorts in a meta-analysis, rather than using them for replication?