But why does this mantra persist; where does it come from?
Behind this line of inquiry is a fascinating observation about the disconnect between the laboratory and the clinic. The basic answer, from my perspective, goes like this:
- It is doctors who talk with people about their urine, and any testing thereof;
- for doctors, urine is practically1 sterile; and
- among those who are aware of the microbiologic reality, doctors inevitably gloss details when explaining laboratory results to patients.
The average clinician learns most of what he knows about his patients' urine from urinalyses. Most clinical urinalysis assays show negative nitrites and negative leukocyte esterase in normal people. A glance at the reference ranges for a clinical assay will demonstrate this. (See also the clinical review below.)
Ideally, urinalyses should not be performed in patients without relevant symptoms, but they are sent anyway, and they're mostly negative.2 Even in people who have symptoms possibly suggesting a UTI they're frequently negative. Furthermore, urine cultures from properly collected specimens grown on the media used in clinical laboratories mostly return at 72 hours: "no growth". This point is emphasized in the abstract by Hilt et. al. linked in the question. Of the cultures that grew in their "expanded quantitative urine culture" protocol,
90% of these specimens were deemed "No Growth" by the standard urine culture technique, highlighting its limitations.
The last phrase "highlighting its limitations" is an interesting one. It appears to me to be yet to be proven that this is a clinically meaningful "limitation" of such assays. Without attempting to refute these data — fortunately this question doesn't require me to do so — I will note that the two linked studies aimed to show that people with overactive bladder are characterized by increased urinary flora. The explicitly stated hypothesis is that this may contribute to symptoms. Even in this study demonstrating the existence of such a micro-biome even in normal people, then, it is understood as a potentially pathologic state.
The reason this "mantra persists", then, is that it remains true for clinical purposes. In contrast to most (all?) other bodily discharges, urine is not packed with bacteria, and the commonly used assays reflect this fact. Medicine and laboratory science have different "modes of discourse", each calibrated to convey levels of precision that are appropriate to the outcomes of interest.
1. In American English at least, that adverb is ambiguous, meaning either "for practical purposes" or (idiomatically) "almost". Both senses are intended here.
2. Of course, there are plenty of data in various populations demonstrating that some substantial minority of urinalyses among asymptomatic people return positive, "proving" that we shouldn't be checking them (amen!). But this is not the point. The gestalt remains: normal = negative.
3. The nitrite parameter in particular is not especially sensitive. It is, however, quite specific (92-100%), meaning it reliably returns negative in people without infection. (See review, below.) Of course, negative nitrites does not mean sterile in the sense that the OP has used it; see "modes of discourse", above.
Simerville JA, et al. Urinalysis: A Comprehensive Review. Am Fam Physician. 2005 Mar;71(6):1153-1162.