http://en.wikipedia.org/wiki/Electromagnetic_pulse mentions the following natural sources of EMP, which I have ordered by frequency / period, from the shortest to the longest:
Electrostatic discharge from objects coming into contact (normally too small to be of any concern)
Lightning (milliseconds)
Solar flares (hours.) Such flares cause geomagnetic storms (more on this later.)
Several artificial sources of EMP are mentioned, of which the only ones of interest are military:
Non-nuclear weapons. The page links to a general article on directed-energy weapons, which mentions microwave devices for causing pain and for targeted damaging of electronic equipment.
Nuclear weapons. http://fas.org/nuke/intro/nuke/emp.htm gives more information on this, listing three phases.
The EMP produced by the Compton electrons typically lasts for about 1 microsecond, and this signal is called HEMP. In addition to the prompt EMP, scattered gammas and inelastic gammas produced by weapon neutrons produce an “intermediate time” signal from about 1 microsecond to 1 second. The energetic debris entering the ionosphere produces ionization and heating of the E-region. In turn, this causes the geomagnetic field to “heave,” producing a “late-time” magnetohydrodynamic (MHD) EMP generally called a heave signal.
The same reference also indicates:
...the region where the greatest damage can be produced is from about 3 to 8 km from ground zero. In this same region structures housing electrical equipment are also likely to be severely damaged by blast and shock.
As far as I am aware, memory loss was not a major recorded symptom among the survivors of Hiroshima and Nagasaki. The only recent data for sufferers of EMP would be from victims of lightning stikes. While the frequency spectrum is admittedly different from nuclear weapons, victims of lightning strikes are surely subjected to greater magnitudes of EMP than those who managed to survive nuclear bombings (who suffered many other traumas, such as blast waves and radiation.) Those who survive the immediate physical (burns) and neurological (cardiac arrest) effects of lightning strikes must surely be knocked unconscious and be disorientated when they regain consciousness. But again, I am not aware of medium or long term memory loss being a particular symptom.
It is worth comparing the structure of the brain (a fairly homogenous mass of tissue of high water content, relatively conductive) with the structure of electronic equipment (components of ever smaller size and greater resistance in order to save power, surrounded by a nonconductive medium such as air, and interconnected by wires, which serve to channel voltage.) It is unsurprising that the brain is less subsceptible to electrostatic damage than electronic components (electronics workers have to take precautions in order to avoid damaging sensitive components with static discharges from their bodies and clothing.)
The greatest damage to equipment caused by EMP is from the long-period pulses, which are channeled by electric cables. These act over distances much larger than the size of the human body.
From the last reference:
...distortion of the geomagnetic field was observed worldwide in the case of the STARFISH test. ... the signal from this process is not large, but systems connected to long lines (e.g., power lines, telephone wires, and tracking wire antennas) are at risk
http://en.wikipedia.org/wiki/Geomagnetic_storm mentions cases of damage caused by long period EMP, including electric shocks suffered by telegraph operators during solar flare events.
So in conclusion, any amnesia caused by EMP is likely to be caused by normal biological shock mechanisms, and not by any direct electrical effect on the brain itself.