Polygenic scores are very good at predicting height. 
This figure, from https://arxiv.org/pdf/2101.05870.pdf, predicts height using common SNPs at r = ~0.65.
In a different study, "combined with sex, a polygenic risk score captured 71.1% of the total variance in adult height in the UK Biobank."
As you can see below, polygenic scores are about as good as the height of your parents at predicting your height.
Yet another study found an r^2 of ∼0.82. In fact, this is greater than some heritability estimates of height.
A good starting point is to understand why Fisher used a linear/additive model. It wasn't just due to convenience or to make the math easier, though that may have been a factor. Fisher's model was empirically driven. He observed that some physical human traits, like height, were normally distributed, and then found that his additive model correctly predicted how similar in height relatives would be. In fact, he incorporated dominance into his models in the 1918 paper, allowing for effects of both dominance and additivity at the same time. The additive model was created to address the shortcomings of dominance models, which already existed at the time.
Fisher said in 1930: "The biometrical facts as to the inheritance of stature and other human measurements, though first regarded as incompatible with the Mendelian system, have since been shown to be in complete accordance with it, and to reveal features not easily explicable on any other view. The approximately normal distribution of the measurements themselves may be deduced from the simple supposition that the factors affecting human stature are approximately additive in their effects."
Forgetting Fisher's result that many complex traits are highly polygenic is what ultimately led to the candidate gene era and its frequent failures.
So why is the additive model so predictive? Is it just a quirk of chance?
Not quite. It is inherent to natural selection. Additive effects are more stable across generations, as they do not depend on the particular combinations of alleles an individual has (unlike dominance or epistasis). This stability makes additive effects more consistently "visible" to selection.
Imagine if a gene for having a long neck was dominant in giraffes--it would be impossible to eliminate the recessive allele from the population (aside from genetic drift), as there is no phenotypic difference between heterozygous and homozygous long-neck individuals. Now imagine if there was simultaneously a different, additive pathway to having a long neck. Selection would favor increasing neck length via the additive pathway, as there would be decreased risk of one's offspring having a short neck. In this case, selection inherently favors additivity.
The second reason why additive effects are biologically preferred is due to the fact that mutations happen randomly. By definition, in order for a new dominant interaction to occur, two mutations must happen. If the mutations require each other to be useful, they'd have to occur simultaneously to be selected for. On the other hand, a new additive mutation being selected for only requires that a single new mutation is beneficial.
The third reason why additive effects are predominant is because most allele frequencies are biased towards one extreme or another. Therefore, the remaining genetic variation between individuals is likely to be of small effect and polygenic.
The final counterintuitive reason is because multilocus epistasis itself mostly contributes to additive variance.
As you saw from the paper you cited, very tall people are not tall because of rare dominant mutations, but because they happen to have a very large number of "tall height" alleles.
Yes, if you take a genome and allow only one SNP to change, there is going to be some change which maximizes the genetic potential for height; if you repeat this with two genomes, the allele will probably be the same unless the individual already has that allele or has a different ancestry than the first individual.
I'm not sure how this means that a linear polygenic score would not fully capture the genotype–phenotype relationship. You are completely correct that SNP-based polygenic scores cannot capture the full heritability: they can only capture the SNP-heritability at best. You are correct that it is possible for alleles to be context dependent. If there is a strong GxE component, and you are measuring between environments, a SNP-based polygenic score's accuracy would be attenuated for the trait.
There are other factors besides this that limit SNP heritability and the maximum accuracy of SNP-based polygenic scores, such as:
- Rare variants (not yet included in many GWAS or polygenic scores)
- Large insertions, deletions, duplications, or rearrangements
- Gene-gene interactions (e.g., dominance): though additivity is more common, dominance is still extremely important, especially for certain Mendelian diseases or non-polygenic phenotypes.
- Copy number variations
TL;DR:
Your intuition is correct, but for many traits like height, linearity happens to be a good assumption due to biological constraints.
P.S.
Since we all learned about dominance in school, it can be surprising to hear that it is really not that common. For example, Okbay et al. 2022 with N = 2,574,253 found that their "dominance GWAS identifies no genome-wide-significant SNPs. Moreover, with high confidence, we can rule out the existence of any common SNPs whose dominance effects explain more than a negligible fraction of the variance in EA." The "combined variance explained by dominance deviations in common SNPs is negligible."
A different study analyzed 70 complex traits on 254,679 individuals in the UK Biobank. The authors "found strong evidence for additive variance," but negligible dominance variance" and "epistatic variance... not significantly different from zero." "Genetic variance for complex traits is predominantly additive." "Epistatic variance is likely to be extremely small in human complex traits." This replicated a similar earlier result that looked at 79 phenotypes. This was replicated again here with 50 phenotypes. Each time, dominance is negligible.
Hill et al. "evaluate the evidence from empirical studies of genetic variance components and find that additive variance typically accounts for over half, and often close to 100%, of the total genetic variance." This is why Crow states "the breeder's custom of ignoring epistasis usually gives a more accurate prediction than if epistatic variance were included in the formulae."
Keightley shows from a biochemical standpoint that polygenicity alone is sufficient for a lack of dominance effects.
Even after accounting for correlations between alleles (LD), "independent of the type and strength of gene interaction, the epistatic variance contributes little to the total."
Twin studies also support the additive model being predominant, with no effect of dominance for most phenotypes.
