Surely an important question. But there are different kinds of memory (classified mainly as declarative and procedural) which you don't specify exactly in your question. Wikipedia and Scholarpedia list here many known facts. I will give you some short hints and links for introduction and overview instead of pasting that stuff here.
You are probably referring to autobiographical or episodic memory, that is memorizing consciously particular events within one's own life within a context. And for this type of memory researchers agree that self-awareness seems to be a necessary precondition. The existence of this cognitive ability can be checked by the mirror test. Human infants pass this test normally around a age of 18 months (also some animals). The cognitive self develops between 21 and 24 months and marks the end of infantile amnesia.