I just had my genome sequenced. Can someone tell me what these different file formats are?

Question

I just had my whole genome sequenced and now I want to play a bit with bio-informatics. However, the documentation is quite scarce on the file standards used.

These are the files I have been given (the "raw" data):

Can someone explain me with some detail what is each file format, and for what are they used?

Transcribed list of file-formats from the picture:

• Indel (TBI)
• Indel (VCF)
• SNP (VCF)
• SNP (TBI)
• CNV (TBI)
• CNV (VCF)
• BAM (BAI)
• BAM
• FASTQ R2
• FASTQ R1

Answer 1

This is a great biological question! It asks a lot about how empirical science is done in the field of modern biology. I'm glad we encourage such questions from curious people who want to learn more.

---

Your genome has a sequence that is unique to you, gifted to you by your parents and by small changes from random chance.

Your genome is pretty close to what is called the reference genome, which gets updated — infrequently¹ — as sequencing platforms, pipelines, and algorithms change.

The reference genome is the "standard bearer". All other individual genomes are compared against it. This lets everyone compare apples to apples when trying to say something about the genome, in the aggregate or "big picture" view.

The files associated with the VCF format specify places in your genome that differ from the reference genome — and how they differ. These differences include:

1. Indels — insertions or deletions of DNA, as compared with the reference.
2. SNPs — single bases that differ from the reference.
3. CNVs — repeated stretches of DNA that differ from repeats in the reference.

The specifics of each of these classes of variants can be associated with various diseases or other attributes that make you unique and interesting. More on that later.

Each of these VCF files in your dataset has an associated TBI file. This is an index file, which helps you to quickly retrieve elements of interest from these datasets using a command-line tool called tabix.

If you are looking for differences within a specific genomic interval — say, all the differences on chromosome 4, from bases 123456 to 567890 — then you can pass that interval to tabix along with your VCF file of interest, and it will return all the indels, SNPs, or CNVs within that range, if there are any within that interval. This subset is VCF-formatted, so by knowing the format, you can get more detail about those differences.

The BAM-formatted file collects all the individual sequencing reads in a form that is mapped or aligned to the reference genome, along with some summary information to tell you how those reads differ from the reference genome.

As with the VCF files, this BAM file has a BAI index file to go along with it. Like tabix, you can use the samtools command-line tool to extract the set of reads over your genomic region of interest, if you want to examine them further.

Both tabix and samtools are available from the same author. If you have questions about these tools, the Bioinformatics Stack Exchange site is a great place to ask questions³. Lots of bioinformaticists there have a great deal of experience with those toolkits and can help answer more specific questions.

The two FASTQ files are the raw sequence data coming off the sequencer, along with quality data. The sequence reads in these two files are paired for higher quality and longer reads, which makes the sequencing platform faster, cheaper, and better.

I walked a bit backwards, here, on purpose.

The raw FASTQ files are starting material for making the BAM file. The BAM file is then, in turn, used with variant calling software (example) to generate the variation (VCF) datasets.

Why walk backwards through these datasets?

Most people don't care about the raw FASTQ data, except perhaps to do manual alignment or check data quality. What is often more interesting or useful to most is how one's genome differs from the reference, which is captured in the variation data.

Those differences, in aggregate, are how companies like 23andme sell a product² that tells you that you are more related to so-and-so population than so-and-so population, or whether you are more or less likely to have some disease or other odd, interesting phenotypic trait.

^{1. Even when new versions of this reference genome are released every few years or so, if you know what original reference genome was used to generate your BAM and VCF files, you can redo analyses based on that knowledge. There is likely some metadata associated with your sequencing results that you can look up to find this out, or you can ask the sequencing center from where you obtained these results.}

^{2. For a bit more context, in this answer to another SE question, I talk a bit about SNPs and how they are used for testing association with so-called Mendelian disorders, diseases caused by single base changes, which may be of interest to you given what motivated you to get your genome sequenced, in the first place: https://biology.stackexchange.com/a/88839/5075}

^{3. You can ask questions about them here, too, but you'll make certain people grumpy. Better to go to the other SE site for that.}