Short answer
The codon-directed non-enzymic binding of tRNA (aminoacylated or not) to the A-site of the ribosome is much weaker than the (normal) binding of aminoacyl-tRNA complexed to EF-Tu/EF-1, the tRNA-binding elongation factor (which discriminates against non-aminoacylated tRNA). Hence it is unable to compete effectively with the latter to disrupt protein synthesis.
Explanation
The stages in protein biosynthesis relevant to this question are shown below.

In stage 1 tRNAs are aminoacylated with their cognate amino acid in a reaction catalysed by a specific aminoacyl-tRNA synthetase.
In stage 2 aminoacylated tRNAs (except the initiator tRNA) are recognized by a single elongation factor (EF-Tu in prokaryotes, EF-1 in eukaryotes) and form a complex with it and GTP. The elongation factor will not form a complex with non-aminoacylated (also referred to as ‘deacylated’) tRNA, and this has been shown to be caused by structural differences from aminoacylated tRNA. This latter is particularly pertinent to the question, and will be discussed in more detail below
In stage 3 this complex binds to the A-site of the ribosome in a codon-specific manner. However it is important to understand the strength of the interaction between the elongation factor and the A-site of the ribosome compared with the tRNA–anticodon interaction alone. The latter is, of course, necessary for accurate protein synthesis, but may be regarded as preventing (or more strictly delaying) dissociation of the complex.
Basis of the discrimination against non-aminoacylated tRNA
The discrimination occurs at the stage of binding by EF-Tu/EF-1. What is known about its structural basis? This appears not to be merely recognation of the amino acid, but involves indirect effects on the structure of the tRNA and its recognition by EF-Tu, as is discussed in the 1996 paper of the Aarhus group that elucidated the structure of the aminoacyl-tRNA.EF-Tu complex. I quote:
Deacylated tRNA binds to EF-Tu-GTP with an affinity which is approximately four to five orders of magnitude lower than that of aa-tRNA. Thus, the aminoacyl group is a primary discriminator in the ternary complex formation. It is impossible to explain this by the direct interactions with the aminoacyl group alone. Other structural features of aa-tRNA must contribute to the affinity.
There has been a long-standing research on the conformational changes of tRNA upon aminoacylation. Fluorescence studies… have indicated that conformational changes occur upon aminoacylation, though in a diverse manner among the individual acceptors.
One precise structural difference is discussed:
It is evident that aminoacylation of tRNA restrains the conformational space of the terminal A76 considerably. Compared to the crystal structure of the ternary complex, the residues A73 through C75 of deacylated tRNA-Phe (PDB entry code 4TNA) are in an equivalent conformation, though slightly shifted in their position relative to the acceptor helix. However, the terminal A76 residue in deacylated tRNAPhe adopts a conformation which is impossible in the phenylalanylated form (fig 7).

[Non-acylated free tRNAPhe (left) and acylated Phe-tRNAPhe of the ternary complex (right)]
It can be seen that the residues 73–76 are near the 3′ end of the tRNA where the phenylalanine (Phe) is attached.
Evolutionary perspective
One presumes that primitive protein synthesis did not involve elongation factors. It would have been important to prevent the unproductive binding of un-aminoacylated tRNA to the ribosome and the competition with aminoacylated tRNA postulated in the question. A first step towards this may have been the evolution of a ribosomal A-site that could discriminate between the structures of aminoacylated and non-aminoacylated tRNA. The development of tRNA-binding elongation factor, as well as making the process more efficient would have amplified this discrimination which is in the range of 10-200x, depending on the concentration of magnesium ions (which artificially enhance binding of non- aminoacylated tRNA).