E. coli has a generation time of about 20 min, and a single CFU will generate a colony a few millimeters in diameter after 16 hours of incubation.
Is this the upper limit? How much faster can bacteria grow? How much faster can colonies grow?
The article and published work here and here respectively, reveals that E. coli replication is close to thermodynamic limits of efficiency having modelled this process using mathematical equations. However an interesting article and published work here and here, respectively reports a bacteria (homologous to Thermobrachium celere and Caloramator indicus, which are closely related) with doubling times as short as 10 minutes have been discovered in areas of hot water emissions in Lake Tanganyika.
Small chimneys emitting hot water at temperatures of 66 to 103 degrees Celsius (150 to 217 degrees Fahrenheit) have been discovered at the shallow lake bottom of Lake Tanganyika, East Africa. In a study searching for thermophilic sulfate reducing bacteria in this extreme environment, a spore-forming bacterium was discovered that grew with a doubling time as short as 10 minutes. In theory, a single cell of this bacterium in this environment can multiply to 500 million in less than 5 hours under optimal conditions (60 degrees Celcius).
EDIT: If the question is exclusively about E. coli and it takes into account laboratory conditions then I also came across this references page claiming tha E. coli doubling time is between 4-20 minutes. Based on this paper the page reports that
Internationally renowned researcher Bernhard Palsson and other researchers from the University of California San Diego led a team of scientists from PNNL, the University of Heidelberg, and the German Cancer Research Center to study how E. coli bacteria changed and evolved from one environment to another in a laboratory setting. The team then compared proteomics, genomic, and metabolic data from the study with a computational model for bacterial growth.
The page then goes on to say
Using high-performance liquid chromatography coupled to mass spectrometry, the team gathered data on how the bacteria changed their protein profile as they grew and reproduced in different mediums. The scientists found that the data and model were correlated to a surprising degree (more than 98% in some cases), validating the model's ability to predict the growth of E. coli bacteria under specific conditions.
This confirms the comments above that the growth rate of E. coli depends on the specific conditions in which it is in and furthermore its growth rate under specific conditions can be predicted based on the devised model.
This 2016 Science blog says that the fastest known bacteria doubling time at the time was Vibrio natriegens, at 2x the E. Coli rate (10 minutes): https://www.sciencemag.org/news/2016/06/scientists-want-replace-lab-workhorse-e-coli-world-s-fastest-growing-bacterium
The blog also explains how better characterization work and tooling might be needed to fully replace E. Coli, but there is promise.