I'm having a bit of difficulty understanding why CpG sites are said to be palindromes. In what sense are they palindromes? Like they always have this shape CGGC? (for example)
CpG motifs are always palindromic. A palindrome is something which reads the same way forwards as it does backwards. A palindromic sequence is a DNA sequence which gives you the same string of nucleotides regardless if you are looking at it on the forward or the reverse strand, so long as you are reading in the same (5' → 3') direction.
Notice that the nucleotide sequence of a CpG island is just CG. So on the forward strand you might denote the sequence 5' CG 3'. On the reverse strand, the sequence is 3' GC 5'. So long as you read both strands in the 5' → 3' direction, you end up with the same nucleotide string: CG. Therefore, because the nucleotide sequence is the same in both the forward and reverse strands, a CpG motif is a palindromic sequence.
An example of a sequence which is not palindromic might be 5' ATT 3' on the forward strand and 3' TAA 5' on the reverse strand. On the forward strand, the sequence is ATT. Reading 5' → 3' on the reverse strand, the sequence is AAT. Because ATT ≠ AAT, this sequence is not palindromic.