1
$\begingroup$

I am using a dataset composed by $m$ samples and $n$ features (genes). Each data point is real number.

I want to understand how to preprocess data before analysis, in particular: do data points follow a normal or a log-normal distribution?

I thought about using qqplots and searching for different tests to assess the form of the distribution, but I have a doubt:

Do I have to assess the form of:

  • each sample distribution
  • each feature (gene) distribution
  • the whole dataset ($m$ samples x $n$ features (genes))

?

$\endgroup$
3
  • 1
    $\begingroup$ This might be better suited for Cross Validated $\endgroup$
    – C_Z_
    Jul 7 '16 at 11:54
  • $\begingroup$ @C_Z_ you are right, but I thought it may be a task well-known for bioinformaticians, e.g. while using microarrays. However, if not, how can I transfer it to CV? Thanks $\endgroup$
    – gc5
    Jul 7 '16 at 12:00
  • 2
    $\begingroup$ Have a look at biology.stackexchange.com/q/37167/3340 $\endgroup$
    – WYSIWYG
    Jul 9 '16 at 8:59
3
$\begingroup$

From personal experience, nearly all count data whether from microarray or reads from RNAseq of some kind, requires a log transformation of the counts. Usually a small fraction is added to all values before doing so to zero protect. Log2(counts + 0.5) or some such. This is independent of the treatments. If you log transform one sample, you will do the same for all samples. To examine for normality, a simple way is to look at the histogram of counts (by all samples or by each sample) before and after transformation. Roughly bell shaped -> proceed.

Pictures below from my data. Although the data are from RNAseq, microarray data should be similar.

R code here:

hist(t$counts,breaks=100,main="Histogram of Raw Counts from RNAseq")
hist(log(t$counts + 0.5,2),breaks=100,main="Histogram of Log2
transformed Counts from RNAseq")

enter image description here

$\endgroup$
1
  • $\begingroup$ There are ways to statistically check for normality, and I find visually examining the data as I listed above is a good first step. $\endgroup$
    – akaDrHouse
    Jul 7 '16 at 15:07
3
$\begingroup$
  • Preprocessing will and should always depend upon the biology that you try to answer or discover (e.g.: There might be an experimental rationale to believe that some genes behave differently in individual samples - and that different samples could possibly have different distributions.)
  • log-transforming your data by itself is usually no problem, and hugely facilitates the simultaneous exploration of different magnitudes (though adding a small value prior log can soon make your analysis misleading, if you intended to quantitatively study variance across samples)
  • For testing normality you might want to apply the Lilliefors test on raw data and log-transformed data
  • If you are using a readout of gene-expression you must not anticipate a unimodal distribution, e.g.: Metazoans have two different classes of genes, which overall lead to a bi-modal distribution (Hebenstreit et al. 2011) (If you can fit any unimodal distribution - such as lognormal - you should become very suspicious and check the quality of the experimental data.)
$\endgroup$

This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .