As I understand it, the main difference between the Bacteria and the Eucaryota domains are that eukaryotes have a nucleus and bacteria don't. I understand that bacteria and archaebacteria have enough of a genetic difference to be separate kingdoms, but why are they separate domains?
The reason that Archaea were determined to be a separate (and only the third) kingdom so late (1977 according to this reference) was because archaea often completely resemble eubacteria. They are unicellular and have no organelles and appropriately they were grouped with other prokaryotes because of their morphology and cellular physiology.
But by the 1970s phylogeny becoming based on the sequence of DNA and RNA which were just becoming more available. Based on a simple comparison of the sequences of 18S rRNA of 13 fungi, bacteria and archaea, Woese and Fox showed that the bacteria and the archaea were about as different from the eukaryotes than from the bacteria. (see table 1).
Since then, the differences between bacteria and archaea have continued to be great. The wikipedia diagram below is not to scale- the divergence of the three kingdoms is so far back that they pre-date the emergence of eukaryotes entirely. But you can see that fungi and other eukaryotes are more similar to archaea than the bacteria.
BTW, the idea that archaea are extremophiles is not entirely true - they are pretty much exclusively the only kingdom found in these conditions, but archaea have been found to coexist with bacteria and eukaryotes in the open sea and their detection heretofore in most environments may be only due to the fact that archaea are difficult to grow in the lab.
Just found this retrospective from Woese himself, who passed away recently.