What is the best practice when preprocessing microarray data using a detection filter (on scanner p-value)?

Suppose I have a microarray dataset that I have to normalize with Loess and correct with ComBat. When should I apply a detection filter, relative to the other steps in the process?

In my experimental design, I have two pipelines to test:

  • Normalize per sample between 0 - 1
  • ComBat
  • Loess normalization between samples


  • Loess normalization between samples
  • ComBat

Is there any best practice for the timing of applying a detection filter?

  • 1
    $\begingroup$ Why not try applying before and after for all pipes and see which gives you more believable answers? $\endgroup$ – virtualxtc Feb 28 '17 at 0:54
  • $\begingroup$ @virtualxtc how do you decide if an answer is more "believable"? You will get a result eventually. $\endgroup$ – gc5 Feb 28 '17 at 10:07
  • $\begingroup$ My guess is that both methods will work equally well, just plot the data and validate by eye. The first rule of biostats (according to Ethan Meyers) is just plot everything & don't add extra thought to it unless l something is broken. That said, I use interquartile normalization (due to the fixed lower bound at 0) then hit call. $\endgroup$ – virtualxtc Mar 13 '17 at 19:48
  • $\begingroup$ did you find an answer to your question? If you did, it would be great if you could post it here for people having similar questions in future $\endgroup$ – user1993 Mar 24 '17 at 7:22
  • $\begingroup$ you might also want to ask the bioinformatics.stackexchange.com $\endgroup$ – prab4th Sep 7 '18 at 8:57

My comment above still stands; provided your detection filter doesn't rely on your data being normalized, your detection filter can exist anywhere.

However, since it seems you know that you want to run a PCA, and know all the normalization's you'd like to try already, it is simplest to run all of your normalization first, that her than having to filter out high scoring vectors that appear to be caused by inter-sample variation.


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.