Phospholipid bilayers found in cells are usually negatively charged. The phosphate groups repel each other by like charges and prevent two membranes from coming too close to each other. Membranes are also full of proteins and often coated with carbohydrates, which serve to keep membranes from interacting too strongly. See this diagram of a membrane bilayer:

Even this diagram is probably cleaned up.
From a more physical chemistry perspective, the energy barrier for flipping a lipid in a bilayer is pretty high, since the polar head group doesn't like passing through the hydrophobic layer. Bringing two membranes together would require overcoming the charge-charge repulsion and flipping around the inner two layers of lipids, so you go from 4 layers to 2.
Interestingly, there are mechanisms for fusing membranes, such as using cationic lipids. These would turn charge repulsion into charge attraction. The lipid ion pairs tend to have shapes that don't support bilayer formation, and tend to prefer micelle or inverted micelle shapes. These have been utilized to break down vesicle membranes and allow large molecules to pass through, like DNA or proteins, and are used in genetic modification of cells.