If a zygote has all the cytoplasmic determinants and all the specific transcription factors, does that mean that all genes in the genome are expressed?
The short answer is that no, a zygote doesn't express all genes simultaneously.
A zygote is like any other cell, in that it represents a distinct phenotype. And like any other cell, this phenotype is maintained by tight regulation of gene expression patterns. Check out this paper, for a nice description of the process of embryonic gene activation.
An important way in which the zygote differs from other cells, and which might be what drives your question, is that the zygote has the potential to differentiate into all the other cell types found in the body. This process of differentiation is also very tightly controlled in terms of gene expression.