LB, TB media, and yeast extract are also yellow but at the heart of the question. What chemical(s) make E. coli yellow?
I think of E. coli being more white than yellow. When you compare E. coli and S. aureus on an agar plate, the Staph colonies have a much more distinctive yellow tint. The yellow tint in Staph colonies is due to staphyloxanthin, a carotenoid pigment.
You will be surprised that when describing the colonial morphology of E. coli on a Blood agar plate, the word choice for colour is "grey". I believe that user leonardo up there had a very good explanation about this "yellowish colour" question.
Bobthejoe's comment is the best answer so far. Despite many other types of bacterial colonies being "more" yellow than E. coli, E. coli is definitely not white.
Flavins, especially riboflavins, are the predominant compound responsible for this coloration.