Without knowing your specific protocol, it is common to use more than one purification method for a single protein. Given the broad similarities that all proteins and even most biomolecules share, it is impossible to completely purify a protein in a single step. The strategy therefore must be to purify via as many properties unique to the target protein as possible until the purity of the sample is sufficient for the desired application. These properties can include most commonly affinity (eg immobilized metal), charge (ion exchange), hydrophobicity (reverse phase) and size (gel filtration).
In a perfect world, a protein could be purified in a single step affinity-based protocol. In the real world, however, non-target proteins can bind the column nonspecifically (for example, binding to the positively charged nickel column or to the agarose matrix itself). Nucleic acids may elute with your target protein or your target may form large, undesirable aggregates. The list of potential problems is long and this is why multiple different rounds a purification can more effectively discriminate between the target protein and other contaminants in the mixture. As an example, let’s say there there is a contaminant binding the Ni-NTA matrix by charge. Using gel filtration afterwards, which separates by size and not charge, allows removal of the contaminant from the target protein.
A major benefit of gel filtration as a final polishing step is that it can also be used as a desalting column. Besides further purifying contaminants from your target protein, gel filtration allows removal of unwanted small molecules (eg imidazole from IMAC) and elution in a buffer desired for downstream applications.
I looked for some further reading in case your interested and found this protein purification handbook which talks about developing purification strategies.