Prokaryotic sigma factors confer specific promoter recognition to RNAP via Sigma subunit σ1.1, which acts as a DNA 'mimic' - occupying the downstream DNA binding site. This interaction is displaced only when the promoter region is bound, hence conferring the specificity.
Note that Sigma factor on its own cannot bind DNA, as σ1.1 binds σ4 in the unbound form, preventing binding to any sequence.
Source: Section 2. Structural Organisation of σ70 and Other Group 1 σ Factors in this paper
This is in contrast to eukaryotic transcription factors, which can bind to the DNA on their own - indeed the assembly of the general transcriptional machinery in eukaryotes (read: In Yeast) is stepwise and involves first the binding of TFIID to the DNA, then TFIIA, then TFIIB, before finally the Polymerase is incorporated into the growing complex. Source: Molecular Biology of the Gene - James D. Watson, p.g.449-454
In prokaryotes, the sigma factor needs to be bound to the core enzyme (forming the holoenzyme) in order for this promoter specificity to be achieved. Your book 'Principle of Genetics - Snustard and Simmons' is referring to the core RNAP, not the holoenzyme with the Sigma factor bound.
This is why sigma factors are often not included under the umbrella term of 'transcription factor' - they achieve the same functionality as eukaryotic TFs, but are fundamentally different in terms of their binding to the polymerase.
Often authors note that eukaryotes need TFs whereas prokaryotes don't because the point they want to convey is that eukaryotes use huge multiprotein complexes bound to the Polymerase, whereas prokaryotes just use the RNAP holoenzyme, giving Sigma factors a somewhat 'special' place among the transcription factors.