If I'm not mistaken only RNA viruses insert themselves into the host genome. As an example of DNA viruses, herpes viruses for example do not insert themselves in the host genome.
Can CRISPR cut DNA that isn't in a chromossome, like a DNA virus?
Yes, this should in principle work, and a number of groups have shows that it works in cultured cells:
We found that CRISPR/Cas9 introduced InDel mutations into exon 2 of the ICP0 gene profoundly reduced HSV-1 infectivity in permissive human cell culture models and protected permissive cells against HSV-1 infection.... Combined treatment of cells with CRISPR targeting ICP0 plus the immediate early viral proteins, ICP4 or ICP27, completely abrogated HSV-1 infection. We conclude that RNA-guided CRISPR/Cas9 can be used to develop a novel, specific and efficacious therapeutic and prophylactic platform for targeted viral genomic ablation to treat HSV-1 diseases.
There's also been at least one trial in an animal model:
Using a hydrodynamics-HBV persistence mouse model, we further demonstrated that this system could cleave the intrahepatic HBV genome-containing plasmid and facilitate its clearance in vivo, resulting in reduction of serum surface antigen levels. These data suggest that the CRISPR/Cas9 system could disrupt the HBV-expressing templates both in vitro and in vivo, indicating its potential in eradicating persistent HBV infection.