12
$\begingroup$

I'm looking for a downloadable list of all known (or better said, online documented) species in this straightforward format, as an example the European Frog:

Kingdom: Animalia
Division: Chordata
Class: Amphibia
Order: Anura
Family: Ranidae
Genus: Rana
Species: Rana temporaria

I'd like to have this list offline in a structured text or database format that I can parse. If such a list has other useful attributes (credits, distribution, common names), it is welcome, but the above is the absolute minimum I need. I know there are more taxonomy levels in the above system (Species 2000 & ITIS Catalogue of Life: April 2013 ), but I only need these basic 6.

I have been looking at several collections and am having difficulty getting the data that I need. Either the database is incomplete, not accessible as a database, has complicated data structures, or has poorly described and slow web APIs that I cannot rely on.

Conceptually, it seems to me that http://www.catalogueoflife.org comes closest to my needs, yet their download is a tool, not a database. I do see the download has a MySQL folder containing lots of files, but I don't know how to reconstruct it into a database. It seems also that they pulled the page to describe the database download.

As simple as my question seems (I'm not a biologist, so I assumed it was simple), I have wasted several hours coming up empty. I was wondering if anybody has done the work to provide basic species info in an understandable format as requested?

As for species completeness, the 1.3m of Catalogue of Life sounds really good, as I'm currently basing my system on Wikipedia, which only has 180k species.

$\endgroup$
5
  • $\begingroup$ you might want to have a look to www.tolweb.org $\endgroup$
    – Remi.b
    Dec 19, 2013 at 22:16
  • $\begingroup$ Have you ever found what you needed? $\endgroup$
    – Rodrigo
    Feb 11, 2017 at 4:53
  • $\begingroup$ Extant only, or all known life? because xtinct life tends to mess with that form of classification, which is why it is included less. . $\endgroup$
    – John
    Feb 11, 2017 at 5:28
  • 2
    $\begingroup$ you have another issue as well, taxonomic classification is less helpful for cross group comparison (some classes for instance are inside other classes) so many larger "tree of life" projects tend toward phylogenetic classification instead, it makes for clearer coding. $\endgroup$
    – John
    Feb 11, 2017 at 5:54
  • $\begingroup$ Second John. Possibly there are some reasons to use Linnean classifications, but the phylogeny is the actually useful data artifact. The hierarchical taxonomy is more of a crude heuristic, sounds like a painful exercise to make it play nicely. $\endgroup$ Mar 8, 2023 at 18:49

6 Answers 6

7
$\begingroup$

ITIS

To expand on the answer from @Rodrigo, the Integrated Taxonomic Information System seems to be a reasonable answer. It gives you a database of every taxonomic node in the Tree of Life™ and each node has a Taxonomic Serial Number (TSN). You can download the data as a database or even just use the tables as pipe-delimited text files.

For example the file vernaculars tells us that the TSN for European frog is 173444. By looking in the file longnames, we can see that the complete name for 173444 is Rana temporaria. Finally, the file hierarchy, tells us that the complete hierarchy for 173444 is:

European frog:
→ Animalia
→ Bilateria
→ Deuterostomia
→ Chordata
→ Vertebrata
→ Gnathostomata
→ Tetrapoda
→ Amphibia
→ Anura
→ Ranidae
→ Rana
Rana temporaria

You may download the ITIS data in several different formats from https://www.itis.gov/downloads.

Example data from some of the files.

If you downloaded the MySQL Tables data from https://www.itis.gov/downloads/itisMySQLTables.tar.gz, here are some of the useful files you'll find.

vernaculars

The first two fields of vernaculars contain the TSN (Taxonomic Serial Number) and the common name for a node. Some common names map to a single TSN and not all TSNs have common names. Some examples:

50|bacteria|English|N|2015-03-02|159942
14092|mushrooms|English|N|2004-01-28|11529
14092|pore fungi|English|N|2004-01-28|11534
14092|rusts|English|N|2004-01-28|11539
183798|Domestic Cat|English|N|2007-08-15|111781

You can read the file dropcreateloaditis.sql to see what the other columns mean.

longnames

The file longnames contains only two fields, the TSN and the complete "scientific" name of the item. For example:

50|Bacteria
14092|Basidiomycotina
183798|Felis catus

hierarchy

The first field of the file hierarchy is the entire hierarchy, from tip to tail, for the TSN in the second field represented as a sequence of TSNs separated by a hyphen (-). E.g.,

50|50|0|0|14789
555705-936287-623881|623881|936287|2|415
202423-914154-914156-158852-331030-914179-914181-179913-179916-179925-180539-552304-180580-552363-180586-183798|183798|180586|15|0

Of course, those TSNs can be looked up in the longnames table to convert to something human readable. Using that, here is what the first field of the last line above actually says,

Animalia → Bilateria → Deuterostomia → Chordata → Vertebrata → Gnathostomata → Tetrapoda → Mammalia → Theria → Eutheria → Carnivora → Feliformia → Felidae → Felinae → Felis → Felis catus

synonym_links

The synonym_links file contains a list of obsolete TSNs and the current TSN which should replace them. For example, the TSN for the vernacular mushrooms is given as 14092 (Basidiomycotina). However, that TSN is not in the hierarchy file. Instead, synonym_links points us to use 623881 (Basidiomycota), which works. Here are some synonym entries:

51|50|2015-03-02
14092|623881|2004-01-28
552455|180586|2007-08-15

Example script

If you have access to a UNIX machine (MacOS may work), you can look up any common name by using a simple script I wrote. It will print a taxonomy, just like for European frog above. You can cut and paste the following commands into your terminal to try it out.

Download the data

    wget https://www.itis.gov/downloads/itisMySQLTables.tar.gz
    tar -axvf itisMySQLTables.tar.gz
    cd itisMySQL*/

Download my lookup.py script

    wget https://tinyurl.com/yyld2mm4 -O lookup.py
    chmod +x lookup.py

③ Run the script giving it a common name

    ./lookup.py human
    ./lookup.py domestic cat
    ./lookup.py love-lies-bleeding

④ Sample output

    human:
     → Animalia
     → Bilateria
     → Deuterostomia
     → Chordata
     → Vertebrata
     → Gnathostomata
     → Tetrapoda
     → Mammalia
     → Theria
     → Eutheria
     → Primates
     → Haplorrhini
     → Simiiformes
     → Hominoidea
     → Hominidae
     → Homininae
     → Homo
     → Homo sapiens
$\endgroup$
4
$\begingroup$

Have you tried NCBI? Try this link:

http://www.ncbi.nlm.nih.gov/taxonomy

$\endgroup$
2
  • 1
    $\begingroup$ Sorry, but there's several problems with this answer. It's a different classification, it only has 10% of species, and as far as I can see, there is no database. $\endgroup$
    – Fer
    Dec 19, 2013 at 0:48
  • 3
    $\begingroup$ @Ferdy: yes there is ncbi.nlm.nih.gov/guide/taxonomy $\endgroup$
    – nico
    Dec 19, 2013 at 14:02
2
$\begingroup$

try this one as well:

http://www.catalogueoflife.org/DCA_Export/archive.php

the document with the taxonomy is the taxa.txt document

$\endgroup$
2
$\begingroup$

Have you tried ITIS?

http://www.itis.gov/

I once used their database. It was in MS Access format.

EDIT: Now they updated their site, and you may download PostgreSQL, MySQL, SQLite, among other formats: https://www.itis.gov/downloads/

$\endgroup$
1
$\begingroup$

I suspect you will struggle to find what you are looking for. However, you could break down your search criteria into smaller chunks to come up with some more successful (but still incomplete) results. For example, species of what? Do you really want ALL species (e.g. Bacteria, Archaea, Eukaryota)? Do you want just species that are currently alive (extant) or all species that are know to have existed at some point? Do you want sub-species?

Once you have decided on such things, you may be better off searching for individual country databases. If you do decide you want ALL species, I suggest you break down each search into smaller chunks - such as 'mammalia'.

$\endgroup$
1
  • 1
    $\begingroup$ He actually don't need to do that. There are a few options to download a full database, as you can see in the other answers. $\endgroup$
    – Rodrigo
    Feb 11, 2017 at 13:59
0
$\begingroup$

If you want Reptile data sets these are downloadable files here is the link http://www.reptile-database.org/data/ or if you want a wider and easier search there is https://www.gbif.org/ you might need to sign in though but in my experience the best data sets for animal species are: GBIF and REPTILE DATABASE

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .