I think this is an interesting question, and I've been struggling to put together a good answer to it for a while. Unfortunately, everything I came up with just boils down to "some viruses are more deadly because they kill more cells faster" which is pretty circular. A detailed understanding at the subcellular level of what happens during virus infections doesn't exist.
It's instructive here to look at the evolutionary pressures on human-infecting viruses. SARS-1 was about 10x as deadly as SARS-2, but it was contained quickly, while SARS-2 is on track to infect a substantial portion of all humans on the planet. SARS-2 is more successful because of, not in spite of, its lower lethality. Because many infections are asymptomatic, the infection can't easily be contained. If we look at things this way, the asymptomatic infections are the ones that go "according to plan" for the virus, and the severe cases and deaths are unwanted collateral damage. Severe cases of COVID are associated with the presence of viral RNA in the blood. But SARS-2 primarily enters new hosts through droplets expelled from infected hosts' respiratory systems. Any viral activity outside of the respiratory system cannot produce these droplets, and is therefore "wasted effort". This is similar to polio, which causes no symptoms when it transmits successfully by replicating in intestinal cells, but causes paralysis when it gets stuck in the dead end of its host's nervous system.
So, if viruses are most successful when they cause no disease, shouldn't they all quickly evolve into harmless versions? They don't because the "sweet spot" of asymptomatic transmission is the unstable result of an immune response that is just effective enough. If a virus fights the immune system either too hard or not hard enough, the infection will end too fast to spread effectively; due to the death of the host or the virus, respectively. Similarly, if a virus is too "aggressive" in entering cells it will damage tissues that don't lead to transmission or new infections, but if it is not aggressive enough it will not be enter those tissues it needs to make, e.g. respiratory droplets carrying new virus in the case of SARS. Since a lethal disease still results in some transmission but complete suppression by the immune system doesn't, it's more evolutionarily favorable for viruses to err on the size of causing worse disease.
Edits for changes in the question:
I think the differences between covid-19 and smallpox are a bit misleading. The big differences between the two diseases is that, while the human populations most affected by smallpox were naive to the virus, the virus was not naive to them. Smallpox is believed to have been infecting humans since at least the 3rd century BCE. It had a lot of time to evolve tricks to defeat the human immune system. And as a dsDNA virus with a 186,000 base pair genome, it had plenty of space to store those tricks. For comparison, SARS-CoV-2 has a 30 kb RNA genome, and poliovirus has a 7.5 kb RNA genome. However, Herpesviruses have been infecting humans since before humans were human, have large genomes, but only cause disease when the immune system is suppressed or nonfunctional. In those cases, however, it is quite effective at spreading throughout the host and can kill. So we see that herpesviruses also rely on the immune system of the host to stay in their sweet spot, they just have a different strategy than smallpox. Since smallpox has been exterminated and herpesvirus is in nearly every human on earth, we can say the mild strategy is the unequivocal winner, at least for now.
