The voice of females is generally of high frequency, while that of males is generally of low frequency.
Also, as we age, we gradually keep losing the ability of hearing sounds of high frequencies.
So, can such a situation occur when a person is not able to hear voices of females, or maybe of certain people, while he can hear the voices of other people? I tried searching about it, but found nothing.
Although you loose the high frequencies in your hearing with age (20kHz and below), this is highly unlikely. The frequency spectrum of our voice is pretty much limited, which is also shown by the fact the for normal telephone, the spectrum transmitted is located between 300 and 3400Hz and the spectrum band between 300 and 3000Hz is named the "voice band". See here for some details.
According to reference 1 the "voice fundamental frequency" for men is between 85 and 180Hz (depending on the reference) and for women between 165 and 255Hz. These frequencies are close enough together to rule out specific hearing of male or female voices. Furthermore the frequencies even overlap.
What happens in age-related pathological hearing loss (also termed Presbycusis) is that voices generally get muffled due to losses in the spectrum and hearing intensity. See the Wikipedia link and also reference 2 for more details.
Reference:
Sensorineural Hearing Loss (SNHL) can reduce the ability to hear faint sounds even when the sounds are clear and audible. Tone deafness in contrast affects a person's ability to learn music, but not the ability to perceive or recognize speech.
