Compared to peaks in terms of disease burden (morbidity and mortality, or incidence of severely symptomatic cases and deaths caused by a viral strain within a population), is the relatively light burden of most endemic respiratory viruses due to
(a) a high level of population immunity
(b) the selection of variants that are less virulent
a: Population immunization. By this I mean evolved resistance. Measles was not a big deal for Europeans. But when they brought measles to Hawaii it killed 80% of the population. Europeans had evolved resistance to measles; a degree of intrinsic immunity. Populations that co-exist with a pathogen for a long time can evolve a degree of intrinsic immunity. There are viruses to which humans have little intrinsic immunity, like Ebola. We have not coexisted with Ebola for long.
b. Variants that are less severe. Nepalese peacekeepers brought cholera to Haiti. Usually with cholera you pour out feces and die, but these soldiers were deemed fit for duty. They had a less virulent strain and they were carriers. That strain reverted to massive virulence in Haiti because they had a transmission advantage there. But in circumstances where sick people are less likely to transmit disease because they are not getting out, less severe variants have an advantage as regards transmission. As regards respirator viruses, this is probably relevant to the common cold - if it leaves you well enough to go to work, you will transmit it.
C: Immunization. Of vaccines US children get, 11 are to respiratory viruses or bacteria. They can be bad diseases if you catch them but because of mass vaccination they are rare. If we stopped vaccinating, they would eventually come back. Except for smallpox, hopefully.