Bacteria or other microorganisms cannot really grow on anhydrous (totally dry) glucose because they need water.
However, they can remain there and cause contamination. Even if you haven't actually touched the glucose sample, there are many bacteria suspended in the air and they may settle down in your glucose packet. When you take the glucose out for weighing, then also bacteria that are in the air, on your gloves, on the paper that you use for weighing, can settle on the glucose sample. In fact any substance, for that matter (salts in PBS, water etc).
That's why before using solutions in cell culture, you have to sterilize them by autoclaving or filtering. Glucose solutions should not be autoclaved because the heat chars glucose. You should filter them using a 0.2 micron filter. All the materials that are to be used for cell culture purposes should be opened only inside the laminar hood.