I know that not all human cells have nuclei (erythrocytes eject theirs before they are fully matured), but do all human cells have all the other organelles? The two I'm particularly concerned with are the Golgi apparatus and lysosomes.
I believe that yes.
Here is more to read about: https://bscb.org/learning-resources/softcell-e-learning/golgi-apparatus/ "is found in all plant and animal cells"
and explanation why so https://sciencing.com/would-did-not-golgi-bodies-8657737.html
I am no expert in this but I was looking for it as well, I believe it is right.