It depends on how much time elapsed between adding the concentrated antibodies to the diluting solution and adding the diluted antibodies to the slide. If it was just a couple of seconds, then I can abolutely see there being a difference. Even if it was longer - say 30-90 seconds, even - there potentially could be antibody gradients in the solution without vortexing. There are a few reasons why. First, many unconjugated primary antibodies that are not intended just for immunostaining are provided in a glycerol solution, with concentrations ranging from 10% or so up to 50% or more, sometimes in addition to a carrier protein like BSA (I used to work for a well-known antibody company, and theirs are in 50% glycerol with 10 µg - 2 mg BSA per ml, depending on the antibody type). The glycerol prevents the solution from freezing solid at -20°C, eliminating ice crystal formation, protecting the antibody protein from degradation and allowing for longer storage times without loss of activity. As a result, however, the solution takes longer to solubilize in PBS or whatever diluent you're using on its own, with no additional mechanical activity.
On the other hand, even if the primary (or secondary) antibody is simply formulated in PBS, the equal diffusion of the proteins to all parts of the tube can take some time. As an experiment, put 1 ml of PBS in an Eppendorf tube, then add perhaps 10-50 µl of a dark dye, such as methylene blue, bromphenol blue, or Trypan blue. Watch it dissociate into the solution over time against a solid white background like a piece of paper. While none of these have the exact same solubility profile as antibodies, it's good enough to show you what happens over the first 30 seconds or so until you can't distinguish the dye anymore.
So, always vortex, even if it's at a relatively slow speed, for a couple of seconds to ensure everything is evenly distributed.