Lots of interesting questions! Let me try to address a few of them as I don't think I am qualified to answer them all but hopefully I can get this thread started. I am a graduate student in the biophysical chemistry field and have been following a little bit of the Crispr Cas9 craze in the last couple of years. So I am not an expert on Cas9 by any means but I do find it interesting.
Many speculate that as long as their have been cellular life, viruses or some obligate parasites have existed as well. This stems from the idea that viruses perhaps are not "non-self" but rather are parts of the host that happen to become inanimate particles that escape the cell and then find another suitable host. Some viruses have co-evolved with the host, perhaps to prevent other viruses from invading its "home/birth mother". Again, much of this paragraph is just speculation, but it is likely that CRISPR systems, or an adaptive immune systems like it have existed as long as viruses/obligate parasites have.
Error rates, you can look at some of the papers that have come out recently but the idea is that there are a lot of error rates, especially if the RNA is constitutively transcribed from a plasmid. Some people have suggested delivering, not the plasmids for the guide RNAs and the Cas9 protein, but instead, the Cas9 protein complexed to the RNA itself. Other ways to control the amount you deliver to a cell include delivering the RNA for the Cas9 protein so it is eventually degraded. This avoids the complication of delivering DNA which can be transcribed multiple times potentially flooding your cell with cas9 protein or guide RNAs, increasing the likelihood of off target effects. Or you can have the cas9 protein and the guide RNA plasmids under inducible promoters, so you only get expression say if you introduce a small molecule to your cultured organism and limit the amount you deliver of it.
As to if there are other systems "superior" to it, and by superior you mean more efficient, and has less off targeting effects? Sure those things may very well exist, since we haven't sequenced the entire planet yet! All joking aside, maybe start looking around at research groups that do adaptive immunity in prokaryotes, you might find some interesting ideas...
VDJ is a very cool system that I know only a little about but this review might be worth checking out:
"Mechanism and Control of V(D)J Recombination versus ClassSwitch Recombination: Similarities and Differences", which can be found here.
Also recombination in meiotic versus mitotic replication is covered in this review: "Meiotic versus Mitotic Recombination: Two Different Routes for Double-Strand Break Repair", which is available here.
So to answer your question, I will have to defer to someone else!
My question to you is what is "advanced"? Life as we understand it has been evolving for about 3-4 billion years. "Advanced" suggests that one organism has evolved better or faster. I don't know if I would call anything more advanced per se, but maybe more complicated and more capable of rapidly adapting to new environmental pressures? Bacteria have us trumped if there were rapid changes to the environment, global warming, asteroids careening into the planet etc.
Estimating this probability requires knowing how large the entire sequence space of the Earth is, some have estimated 5-50 million eukaryotic species but depends on what metric you would use to define a new species. It is unknown for prokaryotes. My guess is that you have 100's of millions of different organisms that are capable of having some type of adaptive immune system that may be inherently different from CRISPR/Cas9 system but still use some similar elements, like nucleic acid complementation, as a mode of defending against obligate parasites. Then the questions are how many of those organisms have we sequenced and how many proteins do we have any idea how they function based on sequence similarities to other proteins? Or if we have directly characterized their 3D structure based on Cryogenic Transmission Electron Microscopy (Cryo EM), NMR or X-ray crystallography. So, I imagine we have just hit the tip of the iceberg in finding natural enzymes that can function as gene editing and gene locating tools. Plus, we may start designing our own enzymes like what Professor David Baker at UW works on. Huge ocean to go exploring, bring a molecular fishing pole :-)
