Sometimes there are identical sequences in Genbank/Genpept, most often coming from the same species but different strains. Ie if I look up "bacteria chitin synthase", the 624-aa protein from Dickeya dadantii shows up three times. There even is an "identical proteins" link.

Is there any filter that I can use to get only one sequence from every group of identical ones? I mean, other than parse downloaded file on my own computer.

  • $\begingroup$ Use the NR database and restrict to the species of interest $\endgroup$ – WYSIWYG Jul 1 '14 at 9:38
  • $\begingroup$ @WYSIWYG you mean the "partially" non-redundant database? :P $\endgroup$ – terdon Jul 1 '14 at 13:45
  • $\begingroup$ What's your end objective? It is trivial to download the sequences and only keep one but would that be OK for you or do you need this to happen on the web site itself? Using RefSeq will bring the duplicates down to 2 for your example but since multiple sequences exist for different strains, it will be hard to avoid them altogether from the NCBI databases. What is your end objective here? There may be better ways. $\endgroup$ – terdon Jul 1 '14 at 13:57
  • $\begingroup$ @terdon.. LOL.. yeah partially non redundant.. $\endgroup$ – WYSIWYG Jul 2 '14 at 4:24

I'm not sure if there's a way on GenBank, but UniProt offers UniRef where you can cluster redundant sequences or specify a lower cutoff (like 90% identity).

  • $\begingroup$ I don't need clusters, I need non-redundant set of sequences, preferably with a proper access (ie reading entire nr to find what I need is too much). But thanks for answer anyway. $\endgroup$ – Synedraacus Jul 1 '14 at 8:05
  • $\begingroup$ well, just take one sequence from each cluster! if you download the UniRef databases, you actually get the non-redundant set. $\endgroup$ – Michael Kuhn Jul 2 '14 at 10:09

I've written a little script to remove identical sequences from fasta to get what I need. To print the list of removed sequences, uncomment line 22

#! /usr/bin/python3
# Removes identical sequences from fasta file

import sys
from Bio import SeqIO
#This is where sequences will be stored
#likely calling str(seq.seq) on every test will be slower
with open(sys.argv[1], 'r') as fasta:
    for seq in parser:
for j in range(0,len(l)-1,1):
        for k in range(j+1,len(l)-1,1):
            if sequences[l[k]]==sequences[l[j]]:
                del sequences[l[k]]
    except KeyError:
for j in sequences.keys():
  • $\begingroup$ This is for offline execution for which many other tools can also be used- EMBOSS or even awk. Wasn't your question about the online search. $\endgroup$ – WYSIWYG Jul 2 '14 at 5:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.