Take the 2-minute tour ×
Biology Stack Exchange is a question and answer site for biology researchers, academics, and students. It's 100% free, no registration required.

Sometimes there are identical sequences in Genbank/Genpept, most often coming from the same species but different strains. Ie if I look up "bacteria chitin synthase", the 624-aa protein from Dickeya dadantii shows up three times. There even is an "identical proteins" link.

Is there any filter that I can use to get only one sequence from every group of identical ones? I mean, other than parse downloaded file on my own computer.

share|improve this question
    
Use the NR database and restrict to the species of interest –  WYSIWYG Jul 1 at 9:38
    
@WYSIWYG you mean the "partially" non-redundant database? :P –  terdon Jul 1 at 13:45
    
What's your end objective? It is trivial to download the sequences and only keep one but would that be OK for you or do you need this to happen on the web site itself? Using RefSeq will bring the duplicates down to 2 for your example but since multiple sequences exist for different strains, it will be hard to avoid them altogether from the NCBI databases. What is your end objective here? There may be better ways. –  terdon Jul 1 at 13:57
    
@terdon.. LOL.. yeah partially non redundant.. –  WYSIWYG Jul 2 at 4:24

2 Answers 2

I'm not sure if there's a way on GenBank, but UniProt offers UniRef where you can cluster redundant sequences or specify a lower cutoff (like 90% identity).

share|improve this answer
    
I don't need clusters, I need non-redundant set of sequences, preferably with a proper access (ie reading entire nr to find what I need is too much). But thanks for answer anyway. –  user3534356 Jul 1 at 8:05
    
well, just take one sequence from each cluster! if you download the UniRef databases, you actually get the non-redundant set. –  Michael Kuhn Jul 2 at 10:09

I've written a little script to remove identical sequences from fasta to get what I need. To print the list of removed sequences, uncomment line 22

#! /usr/bin/python3
# Removes identical sequences from fasta file

import sys
from Bio import SeqIO
sequences={}
#This is where sequences will be stored
#likely calling str(seq.seq) on every test will be slower
with open(sys.argv[1], 'r') as fasta:
    parser=SeqIO.parse(fasta,'fasta')
    for seq in parser:
        sequences[seq.id]=str(seq.seq)
l=list(sequences.keys())
for j in range(0,len(l)-1,1):
    try:
        s=sequences[l[j]]
        for k in range(j+1,len(l)-1,1):
            if sequences[l[k]]==sequences[l[j]]:
                del sequences[l[k]]
    except KeyError:
        pass
        #print(l[j],file=sys.stderr)
for j in sequences.keys():
    print('>',j,'\n',sequences[j])
share|improve this answer
    
This is for offline execution for which many other tools can also be used- EMBOSS or even awk. Wasn't your question about the online search. –  WYSIWYG Jul 2 at 5:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.