Sometimes there are identical sequences in Genbank/Genpept, most often coming from the same species but different strains. Ie if I look up "bacteria chitin synthase", the 624-aa protein from Dickeya dadantii shows up three times. There even is an "identical proteins" link.

Is there any filter that I can use to get only one sequence from every group of identical ones? I mean, other than parse downloaded file on my own computer.

  • $\begingroup$ Use the NR database and restrict to the species of interest $\endgroup$
    Jul 1, 2014 at 9:38
  • $\begingroup$ @WYSIWYG you mean the "partially" non-redundant database? :P $\endgroup$
    – terdon
    Jul 1, 2014 at 13:45
  • $\begingroup$ What's your end objective? It is trivial to download the sequences and only keep one but would that be OK for you or do you need this to happen on the web site itself? Using RefSeq will bring the duplicates down to 2 for your example but since multiple sequences exist for different strains, it will be hard to avoid them altogether from the NCBI databases. What is your end objective here? There may be better ways. $\endgroup$
    – terdon
    Jul 1, 2014 at 13:57
  • $\begingroup$ @terdon.. LOL.. yeah partially non redundant.. $\endgroup$
    Jul 2, 2014 at 4:24

2 Answers 2


I'm not sure if there's a way on GenBank, but UniProt offers UniRef where you can cluster redundant sequences or specify a lower cutoff (like 90% identity).

  • $\begingroup$ I don't need clusters, I need non-redundant set of sequences, preferably with a proper access (ie reading entire nr to find what I need is too much). But thanks for answer anyway. $\endgroup$ Jul 1, 2014 at 8:05
  • $\begingroup$ well, just take one sequence from each cluster! if you download the UniRef databases, you actually get the non-redundant set. $\endgroup$ Jul 2, 2014 at 10:09

I've written a little script to remove identical sequences from fasta to get what I need. To print the list of removed sequences, uncomment line 22

#! /usr/bin/python3
# Removes identical sequences from fasta file

import sys
from Bio import SeqIO
#This is where sequences will be stored
#likely calling str(seq.seq) on every test will be slower
with open(sys.argv[1], 'r') as fasta:
    for seq in parser:
for j in range(0,len(l)-1,1):
        for k in range(j+1,len(l)-1,1):
            if sequences[l[k]]==sequences[l[j]]:
                del sequences[l[k]]
    except KeyError:
for j in sequences.keys():
  • $\begingroup$ This is for offline execution for which many other tools can also be used- EMBOSS or even awk. Wasn't your question about the online search. $\endgroup$
    Jul 2, 2014 at 5:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .