0
$\begingroup$

I'm actually trying to download all the virus protein sequences available in the NCBI database.

I guess the link is this one: ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/viral/

How could I get all the protein sequences in fasta format into one file ? Does someone have codes to do that easily?

Thank you very much.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I believe that this question belongs to StackOverflow, as it's not about biology or related interdisciplinary subjects. Also for such tasks I recommend using Scrapy for Python $\endgroup$
    – Maxim Kuleshov
    Jan 4, 2019 at 17:04
  • 1
    $\begingroup$ @MaximKuleshov — Better on SE Bioinformatics than Stack Overflow as nobody on SO will know anything about the file format of Genbank files. And leave it at that. Recommending a package in a particular language is not appropriate in a comment and not much use anyway if the poster doesn't use the language. $\endgroup$
    – David
    Jan 4, 2019 at 22:01

1 Answer 1

Reset to default
5
$\begingroup$

As with most things, there are multiple ways to do this. Here are two of them:

Download from the NCBI Assembly portal

  • Go to https://www.ncbi.nlm.nih.gov/assembly/ and search for 'Viruses[Organism]'. For our purposes, I am going to restrict this search to Virus assemblies that are latest RefSeq and reference genomes. The query for this will be: "Viruses"[Organism] AND "latest refseq"[filter] AND "reference genome"[filter]. It will return 46 results.
  • To download the protein FASTA sequences for all of these, click on the big blue Download Assemblies button, choose RefSeq as source and Protein FASTA as File Type.
  • Click the Download button and a tarball with FASTA files - one for each assembly - will be created for you to download. You will then have to combine them all into a single file. enter image description here

Download using wget

  • Go to ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/viral and download the assembly_summary.txt file. This is a tab-delimited file with information about every RefSeq viral assembly located in that FTP path. Column 20 has the FTP path for the assembly which you can parse out and use wget to download the protein FASTA file for each assembly, followed by combining them into a single gzipped file as shown below:

    ## download the assembly_summary.txt file
wget ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/viral/assembly_summary.txt
## download the protein FASTA files
grep -v '^#' assembly_summary.txt \
    | cut -f20 \
    | while read -r url ; do wget $url/*_protein.faa.gz ; done
## combine all files into a single one
zcat *_protein.faa.gz | gzip > all_viral_proteins.faa.gz
Improve this answer
$\endgroup$
9
  • $\begingroup$ Hi, thank you for your help, it did work. Unfortunalty when I try zcat *_protein.faa.gz | gzip > all_viral_proteins.faa.gz I get : -bash: /usr/bin/zcat: Argument list too long $\endgroup$
    – Grendel
    Jan 6, 2019 at 20:19
  • $\begingroup$ Finally I found I way by doing : for i in $(ls | grep "_protein.faa.gz");do cat $i >> all_viral_proteins.faa.gz;done $\endgroup$
    – Grendel
    Jan 6, 2019 at 20:24
  • $\begingroup$ Interesting. I never had to concatenate so many files! Is there a reason why you want such a large number of sequences in a single file? Can you not work with multiple files? Presumably, the single file would be very big and may not be the easiest to handle for downstream usage. $\endgroup$
    – vkkodali
    Jan 6, 2019 at 21:50
  • $\begingroup$ It is because I'll make a blast with those sequences as queries, so I'll make a blast database with them. $\endgroup$
    – Grendel
    Jan 7, 2019 at 8:26
  • $\begingroup$ Maybe you can help me again, I would like to download a list of genomes, I have their ID number (list with one ID number by line) and I would like to get all the genomes in .fa format do you know if it is possible with the same script? $\endgroup$
    – Grendel
    Jan 7, 2019 at 13:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .