I had the same thing happen to my leg. I think it eventually went away, but it was definitely many years.
I am going to go with the Tattoo answer. The way tattoos work is that Macrophages in the skin phagocytose the heavy metals in the inks. Because the heavy metals do not interact with Toll-like receptors, or any of the other pathogen receptors that macrophages have on their cell surface, they do not induce a transcriptional signal into the nucleus of the macrophage to induce it to chemotax out of the dermal tissue and into the lymphatic system, so they just sit there. The YouTube channel Smarter Every Day actually has two interesting videos on tattoos Tattooing Close Up and How Laser Tattoo Removal Works.
Macrophages are long lived, so unless they receive transcriptional signals to get them to move out of the tissue they are in, they will just sit there, and if they happened to have engulfed the heavy metals in tattoo ink, or the graphite from a pencil jab, then they sit there with a vesicle filled with that compound. As graphite is pure carbon, I wouldn't worry that much about it, but it is also not not in a form that the body can use, so it will persist. If eventually the macrophages are induced to move, they will likely take their contents with them, which is why tattoos fade over time.
It also is likely that the graphite was taken up by several macrophages in the region, in particles large enough that they cannot be cleared and remain visible, but not so large that the macrophages could not phagocytose them.