I read that "Bacteria are one-celled organisms that can multiply by division", are all one-celled organisms bacteria or are there any more narrow definitions?
Bacteria are one of three domains in the classification of life. You can find more about the bacteria domain at the wikipedia page (It's a long and complex history which is hard to summarise here) and about the domains here. This last defines bacteria loosely consisting of "prokaryotic cells possessing primarily diacyl glycerol (DAG) diester lipids in their membranes and bacterial rRNA, no nuclear membrane, traditionally classified as bacteria."
The composition of lipids in their membrane (DAG-esters) differentiates them from another major class of prokaryotes — Archaea, which have isoprenoid lipids in their membranes.
Initially Archaea was called Archaebacteria. Because of significant differences, including the membrane composition, they are now grouped into a different kingdom.
No, as @rg255 already mentioned, bacteria are one of the main kingdoms of life:
The Bacteria and Archaea are all unicellular organisms (though there are strange Archea like the Pyrodictium genus that are borderline multicellular). The Eukaryota include all plants, fungi and animals but there are also unicellular Eukaryotes. For example, Paremecium is a unicellular animal, brewer's yeast (S. cerevisiae) is a unicellular fungus and Chlorella is a unicellular plant.
In other words, all bacteria are unicellular but not all unicellular organisms are bacteria.