# What is the difference between naive and adjusted p-values in a GWAS study?

What is the difference between a naive p-value and an adjusted p-value in the results of a GWAS study? See from this paper:

After Bonferroni adjustment, a single gene, DCTN4 (encoding dynactin 4) on chromosome 5q33.1, was significantly associated with time to chronic P. aeruginosa infection (naïve P = 2.2 × 10−6; adjusted P = 0.025; Supplementary Fig. 1).

-
Why did they even report it? Because the adjusted isn’t below their significance level? Well, tough luck! – Konrad Rudolph Jul 31 '12 at 18:56
Agree, including the unadjusted p-value is just waving around a big red herring. – Richard Smith-Unna Jul 31 '12 at 19:38
Hummm, do you think they included the unadjusted p-value because the adjusted p-value is not very good? – 719016 Aug 1 '12 at 10:09
No idea. For the normal 5% threshold, the adjusted p-value is actually still significant. I’ve seen several times that an unadjusted p-value was reported in papers without any explanation why that was done. Maybe the researchers just report what the software gives them without questioning whether it makes sense. – Konrad Rudolph Aug 1 '12 at 11:26
Please consider accepting some of the answers to your previous questions. – Fomite Aug 11 '12 at 21:29

There’s an XKCD comic which explains the problem. Unfortunately, that comic is too big to post here. Briefly, a p-value of 0.1 says (roughly) that there’s a 10% chance (0.1) of the observed result being as extreme1 as it is simply due to chance (sampling variation from a population), assuming the null hypothesis is true.

Often, 5% is more or less arbitrarily chosen as “cut-off”: results with p-values below that (which occur by chance 1 time out of 20) are called significant – results above are insignificant.

However, this directly implies that when doing multiple studies, say 20, there’s a very high chance of getting a “significant” result by pure chance, without there being a real effect.

GWAS studies essentially perform experiments on many (hundreds or thousands) of factors simultaneously. Therefore, reporting the pure p-values would be very misleading since there would be a lot of spurious results with “significant” p-values. The probability of getting one or more spurious hits is called the familywise error rate (FWER).

The FWER can be reduced by adjusting the p values, for instance by performing the Bonferroni correction. The resulting p-values are quite a bit higher, which reflects the reduced certainty that the result is due to chance, i.e. the reduced significance.

It doesn’t generally make sense to report the unadjusted p-value. It was probably reported here because correcting for FWER is quite conservative (i.e. it trades a low false-positive rate for an increased false-negative rate) and the authors feared that the adjusted p-value looked less impressive than it should. This involves a fair bit of interpretation.

1 extreme = different from the expected result under the null hypothesis

-
I would have liked to +2 this: +1 for an excellent, precise answer and +1 for xkcdification. – Richard Smith-Unna Jul 31 '12 at 19:36

I'd argue this actually belongs on CrossValidated.

Essentially, the problem is one of how a GWAS study is conducted. By looking over an entire genome for associations, you're actually conducting thousands or millions of experiments, not the single experiment most statistics were designed to handle. As such, you're going to find many results that meet the traditional threshold for "significance" purely by chance.

Early genomics studies had this problem - they'd generate tons of false-positives, and because of the numbers involved, their very small p-values would give an incorrect perception of the precision of their results. One way to handle this is to adjust for multiple comparisons to essentially penalize the p-value to enable you to still use the p < 0.05 threshold.

Another very common choice is to recognize that 0.05 is something of an arbitrary boundary, and simply choose a much, much stricter criteria for significance. This is, as far as I'm aware, what most state of the art GWAS epidemiology studies do.

-
Indeed. Bonferroni p-value correction simply divides the desired confidence (usually 95%: 0.05) by the number of independent tests performed - this gives you an adjusted 95% cut-off for significance. Alternatively, the Benjamini-Hochberg method actually adjusts the individual p-values after you have performed all your tests (see the "False discovery rate" wiki en.wikipedia.org/wiki/False_discovery_rate) – Luke Aug 13 '12 at 15:17