No, replication cannot happen in the absence of polymerase (on the timescales relevant to humans).
You are correct that in PCR, the first step is to separate DNA strands at 98 C. This heating dissociates the strands semi-permanently. So when you cool down the reaction, the strands do not re-associate very quickly.
When you cool down the reaction to 72 C, A/T and C/G can make temporary connections with their opposite bases. However, the free nucleotides cannot quickly form phosphodiester bonds with the growing chain of DNA in the absence of a polymerase. The nucleotide will most likely dissociate from the DNA strand before the correct bond forms. In other words, it is a kinetic problem.
However, at high temperatures, specialized conditions, and on long timescales, such as those present in earth's pre-biotic period, the answer may be different.