I'm looking for papers quantifying the validity of ontologies such as GO.

I'm relatively new to using annotation terms, and I often hear things like "annotation is messy" and "GO is good but it's not perfect." So I'd like to find papers that give numbers for just how messy.

I'm looking for answers from any experts that have a sense of the "quality" of GO terms (however you want to quantify it). For example, what fraction of terms are just wrong? The more computational parts of me want to sum up GO (or other ontologies) with a few descriptive validation statistics. e.g. "GO terms have an 80% precision and an R^2 of 0.6."

I'd like to see a few more examples similar to this paper that shows that a good fraction of GO ontology can be rebuilt from the ground up from independent data. That seems like one way to validate GO quality. They include this line about lack of GO validation in general literature: "... problems [with consistency and bias] are difficult to assess due to the lack of any definitive gold standard for the rigorous validation of GO."

  • $\begingroup$ for what did you planned to use GO annotation? genes or proteins( maybe enzymes?) $\endgroup$ – KingBoomie Dec 1 '16 at 9:36
  • $\begingroup$ @RickBeeloo Genes or proteins would be good. Although, really, any attempts to quantify bias or precision of annotation terms would be interesting. $\endgroup$ – R Greg Stacey Dec 1 '16 at 18:43

There are quite a lot of articles(1,2, etc.) which have investigated this, I think this article did a great job to give a visual representation of the specificity, reliability and coverage:

enter image description here

also this article noticed the following:

Compared to electronic annotations, it is not surprising that curated annotations have a considerably lower average coverage (Fig. 8, vertical lines). Indeed, the main appeal of electronic annotations is precisely that they scale efficiently to large quantities of data. But in terms of reliability, and contrary to current beliefs, curated annotations that use evidence other than experiments from primary literature do not fare better than electronic annotations (Fig. 8, horizontal lines, Fig. S9 in Text S1). In fact, we observed a higher reliability for electronic annotations than for curated annotations (0.52 vs. 0.33). the figure blow is figure 8 from the article enter image description here

I don't know if you are just curious in the reliabilty of GO or if you are searching for an other kind of hierarchical annotation method, but e.g. for enzymes there are also EC numbers. I can continue citing the article but I would suggest to read the article from which I used the figures ( Quality of Computationally Inferred Gene Ontology Annotations). Secondly this article is more recent and something similair was done here. Among other things they also compared the computional annotation from 2008 with manual annotation in 2010 for the same set of GO terms:


So I would also recommend to read The what, where, how and why of gene ontology—a primer for bioinformaticians. This article and the previous mentioned article will give you a good insight in GO terms.

  • $\begingroup$ No problem, I think the articles will make it clear for you $\endgroup$ – KingBoomie Dec 1 '16 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.