What is the difference between a naive p-value and an adjusted p-value in the results of a GWAS study? See from this paper:
After Bonferroni adjustment, a single gene, DCTN4 (encoding dynactin 4) on chromosome 5q33.1, was significantly associated with time to chronic P. aeruginosa infection (naïve P = 2.2 × 10−6; adjusted P = 0.025; Supplementary Fig. 1).
There’s an XKCD comic which explains the problem. Unfortunately, that comic is too big to post here. Briefly, a p-value of 0.1 says (roughly) that there’s a 10% chance (0.1) of the observed result being as extreme1 as it is simply due to chance (sampling variation from a population), assuming the null hypothesis is true.
Often, 5% is more or less arbitrarily chosen as “cut-off”: results with p-values below that (which occur by chance 1 time out of 20) are called significant – results above are insignificant.
However, this directly implies that when doing multiple studies, say 20, there’s a very high chance of getting a “significant” result by pure chance, without there being a real effect.
GWAS studies essentially perform experiments on many (hundreds or thousands) of factors simultaneously. Therefore, reporting the pure p-values would be very misleading since there would be a lot of spurious results with “significant” p-values. The probability of getting one or more spurious hits is called the familywise error rate (FWER).
The FWER can be reduced by adjusting the p values, for instance by performing the Bonferroni correction. The resulting p-values are quite a bit higher, which reflects the reduced certainty that the result is due to chance, i.e. the reduced significance.
It doesn’t generally make sense to report the unadjusted p-value. It was probably reported here because correcting for FWER is quite conservative (i.e. it trades a low false-positive rate for an increased false-negative rate) and the authors feared that the adjusted p-value looked less impressive than it should. This involves a fair bit of interpretation.
1 extreme = different from the expected result under the null hypothesis
I'd argue this actually belongs on CrossValidated.
Essentially, the problem is one of how a GWAS study is conducted. By looking over an entire genome for associations, you're actually conducting thousands or millions of experiments, not the single experiment most statistics were designed to handle. As such, you're going to find many results that meet the traditional threshold for "significance" purely by chance.
Early genomics studies had this problem - they'd generate tons of false-positives, and because of the numbers involved, their very small p-values would give an incorrect perception of the precision of their results. One way to handle this is to adjust for multiple comparisons to essentially penalize the p-value to enable you to still use the p < 0.05 threshold.
Another very common choice is to recognize that 0.05 is something of an arbitrary boundary, and simply choose a much, much stricter criteria for significance. This is, as far as I'm aware, what most state of the art GWAS epidemiology studies do.
