Are "constraint trees" in phylogenetic analysis cheating? How can I correctly use them?

Question

How do phylogenetic experts evaluate results which use constraint trees? Are there examples in the literature where phylogenetic experts detail how they use constraint trees?

Here is my problem. I have 500 samples. These are five different "categories" of species A, B, C, D, E, 100 samples each.

Category A: A1, A2, A3, ..., A100
Category B: B1, B2, B3, ..., B100
Category C: C1, C2, C3, ..., C100
Category D: D1, D2, D3, ..., D100
Category E: E1, E2, E3, ..., E100

There are two classes of which these "categories" fit in: Class 1: A and B, Class 2: C, D, and E.

For the purpose of this discussion, let's say A=whale, B=dolphin, C=mouse, D=gerbil, E=hamster

When I do the analysis, there are some problems. The output tree resolves the samples into two classes which meet my expectations; it splits the samples into two groups: ( A & B) and ( C & D & E ). That's great. However, it doesn't resolve the species into separate categories, e.g. it mixes together "whale" samples with "dolphin" samples, and mixes together all the individual C, D, E . I see

(A1, B3, A27, B23, B2, A2, A3, A4, ....), (C3, C5, D72, D3, E1, E2...). 

This would mean whale1 is more evolutionarily "related" to dolphin3 than whale 27.

I try to use a constrain tree instead to constrain each of these groups, using the form

((A1, A2, ..., A100), (B1, B2, ..., B100), (C1, C2, ..., C100), (D1, D2, ..., D100), (E1, E2, ..., E100))

Sure enough, my tree now resolves each "category" A, B, C, D, E into a distinct category.

Question 1: Isn't this cheating? I feel like I stipulated "resolve the tree in this manner I expected" and RAxML output that.

Question 2: How do I evaluate whether this is correct?

Question 3: For experts, could direct me to published literature which used this approach? Which papers used RAxML constraint trees?

Thanks for any help. I'm happy to clarify things.

Answer 1

I understand where you are coming from when you say that it feels like cheating. However, there are a coupled of things to keep in mind. First let's consider why your tree topology might not reflect concordance with intuition/species trees:

1) Presence of artefacts such as long-branch attraction

2) Lack of sufficient phylogenetic signal/presence of non-phylogenetic signal (due to, for example, convergent evolution in the sequences)

3)Horizontal gene transfer (unlikely in animals).

4) Incomplete lineage sorting/deep coalescence (hemiplasy).

Only in the case of 3 and 4 does your tree reflect true relationships among taxa that genuinely differ from the species tree. Cases 1 and 2 are artefactual. In cases 3 and 4, it would not be appropriate to use a constraint tree to get a species-tree concordant topology because the resultant topology would not reflect the 'true' relationships among taxa (i.e. would be 'cheating'). In cases 1 and 2, I would argue that it is appropriate to use a constraint tree to get the species-tree concordant topology (depending of course on what you want to use the tree for) because you know that the your current topology is species-tree discordant because of artefacts and is therefore not reflective of the 'true' relationships among taxa. Of course, you are unlikely to know a priori what factors (1-4) are causing your bad topology (though you can do some tests for long-branch attraction for example -- see Cladistics 21 (2005) 163–193).

What you can do is test whether the log-likelihood of your non-constrained tree and your constrained tree differ significantly. You can use tree topology tests such as the Kishino and Hasegawa (KH) test. I don't know about RaxML, but IQ-TREE has excellent topology testing: http://www.iqtree.org/doc/Advanced-Tutorial. If your constrained topology is significantly different (statistically) from the ML topology, then you probably should not use the constrained tree. I am by no means proficient in phylogenetics concepts so I would take my advice with a grain of salt, hopefully some of the other members can illuminate the issue a bit further.