What were the challenges to sequencing the last 8% of the human genome that took 20 years to overcome and how was this done? (T2T Consortium)

Question

CNN's March 31, 2022 article Scientists sequence the complete human genome for the first time says:

In 2003, the Human Genome Project made history when it sequenced 92% of the human genome. But for nearly two decades since, scientists have struggled to decipher the remaining 8%. Now, a team of nearly 100 scientists from the Telomere-to-Telomere (T2T) Consortium has unveiled the complete human genome -- the first time it's been sequenced in its entirety, the researchers say.

and

The research, published in the journal Science on Thursday, was previously in preprint, allowing other teams to use the sequence in their own studies.

Until now, it was unclear what these unknown genes coded.

"It turns out that these genes are incredibly important for adaptation," Eichler¹ said. "They contain immune response genes that help us to adapt and survive infections and plagues and viruses. They contain genes that are ... very important in terms of predicting drug response."

Eichler also said that some of the recently uncovered genes are even responsible for making human brains larger than those of other primates, providing insight into what makes humans unique.

This remaining 8% of the human genome had stumped scientists for years because of its complexities. For one thing, it contained DNA regions with several repetitions, which made it challenging to string the DNA together in the correct order using previous sequencing methods.

The researchers relied on two DNA sequencing technologies that emerged over the past decade to bring this project to fruition: the Oxford Nanopore DNA sequencing method, which can sequence up to 1 million DNA letters at once but with some mistakes, and the PacBio HiFi DNA sequencing method, which can read 20,000 letters with 99.9% accuracy.

CNN's explanation is helpful as far as it goes, but with Nanopore's error rate of 1% to 3% (1, 2) and the difference between the repeats of some repeated sequences presumably a lot less than that, how was the error-prone Nanopore method combined with the HiFi method to nail the last 8% of the human genome accurately despite the repeated sequence challenges?

Question: What were the challenges to sequencing the last 8% of the human genome that took 20 years to overcome and how was this done? (T2T Consortium)

A premise of the question post is that it was the presence of repetitive sequences that posed the primary challenge, but the CNN article says "...because of its complexities. For one thing, it contained DNA regions with several repetitions..." which suggests there are other important factors as well.

Update: From the March 31, 2022 item in Science Most complete human genome yet reveals previously indecipherable DNA:

In six papers in Science, the Telomere-to-Telomere (T2T) Consortium—named for the chromosomes’ end caps—fills in all but five of the hundreds of remaining problem spots, leaving just 10 million bases and the Y chromosome only roughly known. And today, the T2T consortium announced in a tweet it had deposited a correct sequence assembly of the missing Y.

I think that of the six, the main paper is The complete sequence of a human genome

---

¹Evan Eichler, a Howard Hughes Medical Institute investigator at the University of Washington and the research leader, said Thursday.

Answer 1

The ~8% of the sequence that was missing was, as you say, complicated by high repeat content. Repeats make the problem of computational genome assembly hard at the best of times. Long repeat arrays make the same problem impossible unless you have sequencing technologies that have reads at least as long as the arrays in question.

A non-genomics example

Consider a sentence in human language where single words or phrases are heavily repeated. An example from German would be:

Wenn hinter Fliegen Fliegen fliegen, fliegen Fliegen Fliegen nach.

This is a grammatically correct and meaningful sentence that happens to have a "repeat array" of 6 copies of the word fliegen. But it can be confusing when the brain has trouble tracking that much information. Imagine that you only caught the beginning and the end, but otherwise you just know that the word "fliegen" was in there a bunch in that repeat array. How would you figure out what is going on? You just know:

Wenn hinter Fliegen Fliegen....

...Fliegen Fliegen nach.

Your understanding of the sentence is "anchored" in the unique words on either side of the fliegen array. But you have no idea what's going on in the middle there.

How you solve this problem is by paying close attention and remembering the sentence in its entirety. You may not be doing this habitually in casual conversation, but a highly motivated person who knows the language should not have any issues with this.

How this is related to genomics

Genomics technologies got very good at reading DNA a little bit at a time. At the time of the publication of the draft human genome originally in 2001, the predominant method of sequencing had an average read length of ~500 base pairs. A chromosome on the other hand is on the order of 100 million base pairs. This is ok from a genome assembly perspective without repeats. Even with repeats it is asymptotically ok for naive assembly as long as your repeat arrays are shorter than your longest reads. In other words, if your reads can pay attention through the whole repeat array, you can probably assemble the repeat array into the genome.

People got very clever with tools like BACs and YACs to get up to a million or so base pairs, though those tools still suffer from a lot of limitations and are quite labor intensive.

Now, consider that over half of the human genome is repeat. When we got lucky we were able to assemble human genome repeat arrays of around a million base pairs or so, but not in every case. Wherever we failed to assemble through such a repeat array, or where we simply were unlucky for some other reason, there exists a "gap" in the human genome.

In some cases we know what's on either side of the gap (e.g. it is in a chromosome), but not what's in the middle. In 2001 there were over 100,000 gaps with a total length of 80M bp in the human genome. That's a lot of missing stuff. In more recent genome builds it is actually higher (~5%), mostly because we simply ascertained more genome sequence since 2001.

In other cases we don't know what's on the other side of a gap, and these sequences are properly called "contigs"; these are little scraps of sequence that we weren't able to put into a chromosome at all, that we know exist somewhere but can't place properly. (There are some technical quirks here in that there anchored contigs and unanchored contigs, and a whole taxonomy of different flavors of not knowing stuff about sequence, but we don't need to get that down in the weeds!) The current standard build of the human genome, which improved dramatically on the 2001 publication, still has ~400 such contigs.

So there are lots of repeats in the human genome that were very hard to bridge. Up until now, the human genome has had a lot of Wenn hinter fliegen fliegen...??? without being able to read through the whole sentence.

Long read sequencing closes gaps

The T2T consortium is different in that it used "long read sequencing" of two kinds, nanopore ("ONT") and PacBio HiFi.

Nanopore reads are huge. They can get up to 1 million base pairs for very good runs. That's just enormously longer than methods like Sanger (~1000bp max) or current workhorse Illumina (~500bp max). You can read through tons of stuff with reads that long that was simply impossible.

The downside of nanopore is that you have somewhat low confidence in the accuracy of any given base, but on average it is still good enough to computationally take the average of a bunch of reads and get a good idea of what the genome looks like. Nanopore is great at stitching together problematic regions separated by repeat arrays, by just blowing through the whole array by brute force. HiFi is higher confidence but shorter (a mere ~20K bp per read!!) and much higher volume, so it is great for building the chunks to stitch together with nanopore.

Conclusion

The change of the T2T consortium was chiefly a technological change that allowed us to read through repeat arrays. People had previously used a lot of clever algorithms to bootstrap short reads, but they were still fundamentally limited by short reads failing on big repeat arrays. T2T removed that limitation, such that our long reads are now long enough to read through basically everything in the human genome.