The data processing inequality (DPI) states that if $X\rightarrow Y \rightarrow Z$ is a Markov process, then $I(X;Y) \geq I(X;Z)$. We can think of $X$ as a random variable representing some complex functionality that is unrelated to the organism's reproductive fitness, e.g. high intelligence in humans.

If $f(Y) = Z$, and $f$ is a standin for evolution, then evolution can only increase mutual information between $Y$ and $X$ if $f$ contains information about $X$.

However, evolution only contains information related to reproductive fitness. Therefore, $f$ does not contain information about $X$, since $X$ is unrelated to reproductive fitness.

According to the DPI, $f$ should decrease the mutual information between $Y$ and $X$.

This means that the process of evolution should eliminate all functionality that is not directly related to reproductive fitness.

Yet it seems many organisms have very complex functionality that is not related to reproducing. Ironically, the higher up the food chain we go the less reproductive the animals become. So, based on this argument, much of the upper echelons should not exist. What am I missing here?


Here is a simple program in python that illustrates the point.

  1. Measure fitness by number of 1s in the bitstring, more 1s the more fit it is.
  2. Set a random bitstring to be a phenotype of interest, P.
  3. Set another random bitstring to be the organism being evolved, O.
  4. Iteratively evolve the organism by flipping a random bit, and then keeping the new bitstring if it is more fit, otherwise rejecting it.

As long as there is no correlation between the phenotype and the fitness function, then evolution cannot turn O into P.

from random import random

P = [int(random() > 0.5) for i in range(100)]
O = [int(random() > 0.5) for i in range(100)]

print "Matches between O from P", sum([o == p for o,p in zip(O,P)])

for i in range(1000):
    j = int(len(O) * random())
    old_fitness = sum(O)
    O[j] = (O[j] + 1) % 2
    new_fitness = sum(O)
    if old_fitness > new_fitness:
        O[j] = (O[j] + 1) % 2

print "Matches between O from P", sum([o == p for o,p in zip(O,P)])
  • 2
    $\begingroup$ I didn't understand all the maths, however, you have to give examples of "complex functions not related to survival and reproduction" ... All biological functions are either very useful to survival and multiplication, else they are relics of previously apt designs, which are vestigial or modified. ... Also, you will see that mammals can have small bloodcells without DNA nuclei, and small veins, that means they can miniaturize and be metabolically apt. Mammals radiated into bats, whales, moles because of their big "dormant" genome. Birds and lizards, with small genomes and bloodcells, didn't. $\endgroup$ Oct 13, 2017 at 12:19
  • 3
    $\begingroup$ Your paragraph beginning "However, evolution is not purposeful" is gibberish. There's no connection between the first sentence and the second. I can't tell if you simply misunderstand evolution or if you've made some logical fallacy in your assumptions, but I suggest you focus on that and explain why you think this makes sense. $\endgroup$
    – iayork
    Oct 13, 2017 at 13:04
  • $\begingroup$ All of this in a neutral, respectful tone: ".. if $f$ contains information about $X$. -- $f$ doesn't "contain" anything; it's just a mapping. Also, $f(Y) = Z$ doesn't say anything about $X$, and, it's quite erroneous to just throw in evolutionary processes like that, most especially without even attempting to provide a definition of $f$, other than "$f$ is a standin for evolution". $\endgroup$
    – user22020
    Oct 13, 2017 at 17:39
  • $\begingroup$ "According to the DPI, $f$ should decrease the mutual information between $Y$ and $X$." -- though this may be true of Markov chains in general, when applied to biology, it falls apart. Not all evolutionary traits undergo divergent changes, and, it could be that, due to the environment of the organism, $X \rightarrow Y$ could be reinforcing a lack of change between states, and thus, entropy between the two variables is minimized over time, not maximized. $\endgroup$
    – user22020
    Oct 13, 2017 at 17:47
  • 2
    $\begingroup$ Overall, it's great that you're attempting to describe biology within such a probabilistic context, as that way of modeling can be fairly dependable depending on the question(s) being asked. The problem though is that the probability & mathematics that you're studying is far too complex to accommodate such superficial logical leaps as the ones you're attempting to make. That being said, I am VTC due to OP being too broad (however, I think it could also fall within being primarily opinion based, since your proposed conclusions really aren't based on anything concrete). $\endgroup$
    – user22020
    Oct 13, 2017 at 18:12

2 Answers 2


You are misinterpreting the meaning of that inequality and confusing information and entropy (there are other logical errors but this one is the first and most obvious, and makes all the others unimportant by comparison).

$I(X;Y) \geq I(X;Z)$

is referring to the mutual information between X and Y compared to the mutual information between X and Z. Essentially "knowing X, how much can I reduce my uncertainty about Y or Z". Those quantities say nothing about the actual magnitude of entropy or complexity in X, Y, or Z, except for placing a lower bound (i.e., you can't have more information about X from Y than the total amount of entropy in X or Y). Entropy in Z could potentially be much greater than entropy in X, but the inequality is just saying you can't get any more information about Z from X than you can from Y.

Interpreting X, Y, and Z as DNA codes, this inequality is perfectly consistent with one part of the evolutionary process that you seem to be thinking about: the introduction of changes into the genetic code (mutation). $I(X;Y)$ in this context would be a measure of how similar the genomes of X and Y are. If Y evolves from X and Z evolves from Y by purely random processes (we are omitting selection and other processes here, which is of course an incredibly important part of evolution as well), then you would expect that inequality to hold: X and Y would be more similar to each other than X to Z. Because evolution is not strictly a random process, however, you could actually get cases where for small parts of the DNA the inequality would not hold. That is because selective pressures do contain information. At the whole genome level this would be probabilistically impossible, however.

You could actually use these measures as a rough way of estimating relatedness (though there are better purpose-built measures for doing so). Essentially, if $I(X;Y) \geq I(X;Z)$ you could conclude that it is likely that X and Y are more closely related species than X and Z.

If you want to view X, Y, and Z as phenotypes, there is really no difference. Decreases in mutual information with X from Y to Z doesn't mean a loss of functionality as you imply, it only means that X and Z are more different than X and Y.

  • $\begingroup$ "Essentially "knowing Y or Z, how much can I reduce my uncertainty about X"." -- This is an incorrect interpretation, given the context of a Markov chain. In a Markov chain, as you may or may not already know, only immediately future events are attempted to be predicted from the current state of the system. And so, when utilizing mutual information inequality, the verbiage would be, "Given X, how much can I say about Y? And, given Y, how much can I say about Z? And then, because X implies Y implies Z, can X say anything about Z?" The perspective of comparison is always forward moving. $\endgroup$
    – user22020
    Oct 13, 2017 at 17:12
  • $\begingroup$ "Those quantities say absolutely nothing about the actual magnitude of entropy or complexity .." -- This statement is inaccurate. $X, Y,$ and $Z$ must be measurable variables/functions, and so, $I(X;Y) = \sum_{y\in Y}\sum_{x\in X}p(x,y)lg(\frac{p(x,y)}{p(x)p(y)}) = H(Y) - H(Y|X)$ does contain entropy information, where H() is entropy function. See this diagram as well. $\endgroup$
    – user22020
    Oct 13, 2017 at 17:13
  • $\begingroup$ "Interpreting X, Y, and Z as DNA codes, this inequality is perfectly consistent with evolution. " -- This is not true either. In a Markov chain, the future state of the system must purely be dependent upon the immediately local, current state of the system, and nothing else. In this sense, evolution does not match well with an ideal Markov chain, since evolution of genes is governed by many forces completely external to the interval of DNA code; i.e., a forest fire will occur regardless of the DNA makeup present within the organisms subject to experiencing the fire. $\endgroup$
    – user22020
    Oct 13, 2017 at 17:19
  • $\begingroup$ You really misread my answer. To your first comment: "Essentially "knowing Y or Z, how much can I reduce my uncertainty about X" is talking about information theory; there is no directionality to Shannon information. To your second point, you missed the next statement: The information sets a lower bound on the entropy, but that's it. Look at the equation you just wrote: I(X;Y) = H(Y) - H(Y|X). I(X;Y) can be zero even if H(Y) is very large, as long as H(Y|X) is similarly large. $\endgroup$
    – Bryan Krause
    Oct 13, 2017 at 18:19
  • $\begingroup$ For your third comment, again, I'm not talking about Markov chains or their connection to evolution, I am talking about the inequality the OP was using. I didn't say Markov chains are a good model for evolution, I said that the inequality is consistent with evolution. I open my answer making it clear that I don't intend to dissect all of the logical missteps in the question, but rather I focus on one upon which the rest of it falls apart. $\endgroup$
    – Bryan Krause
    Oct 13, 2017 at 18:21

I think this is missing the elephant in the room, namely that you need to have selection (natural or otherwise) for evolution to happen (in a reasonable time). And selection (unlike mutation) is not a local operation on the DNA strands as required by the data processing inequality. Selection tosses out some individuals carrying certain strands in relation to others carrying different strands. The "comparison" performed by selection is not a local post-processing operation (as required by DPE to apply) on a single strand/individual; it takes two or more things to compare them...

The more relevant math for evolution is the Price equation, which partitions the change in fitness in that due to selection and transmission causes (like mutation, recombination etc.) If there's no selection, you're left just with the transmission causes. And in your case the single-individual/strand post-procession would be just mutation. What your math is saying is that not much evolution is likely to happen by mutation alone. (Also, you are taking evolution to mean an increase information/complexity rather than in fitness, but that is a lesser sin. An elephant with 500 legs isn't necessarily the most fit in many environments.)

Given your stated interest in ID, I have the feeling you're dressing up with math a common misconception/argument used by ID proponets, namely that complexity is unlikely to occur via evolution; usually the ID propoents jump to that conclusion by misrepesenting (or misunderstanding) the evolutionary mechanims (as you have done here). Your question reminds of this common ID argument:

Assume that, at each mutational step, there is equally as much chance for it to be good as bad. Thus, the probability for the success of each mutation is assumed to be one out of two, or one-half. Elementary statistical theory shows that the probability of 200 successive mutations being successful is then (½)^200, or one chance out of 10^60. The number 10^60, if written out, would be "one" followed by sixty "zeros." In other words, the chance that a 200-component organism could be formed by mutation and natural selection is less than one chance out of a trillion, trillion, trillion, trillion, trillion! Lest anyone think that a 200-part system is unreasonably complex, it should be noted that even a one-celled plant or animal may have millions of molecular "parts."

The evolutionist might react by saying that even though any one such mutating organism might not be successful, surely some around the world would be, especially in the 10 billion years (or 10^18 seconds) of assumed earth history. Therefore, let us imagine that every one of the earth's 10^14 square feet of surface harbors a billion (i.e., 10^9) mutating systems and that each mutation requires one-half second (actually it would take far more time than this). Each system can thus go through its 200 mutations in 100 seconds and then, if it is unsuccessful, start over for a new try. In 10^18 seconds, there can, therefore, be 10^18/10^2, or 10^16, trials by each mutating system. Multiplying all these numbers together, there would be a total possible number of attempts to develop a 200-component system equal to 10^14 (10^9) (10^16), or 10^39 attempts. Since the probability against the success of any one of them is 10^60, it is obvious that the probability that just one of these 10^39 attempts might be successful is only one out of 10^60/10^39, or 10^21.

All this means that the chance that any kind of a 200-component integrated functioning organism could be developed by mutation and natural selection just once, anywhere in the world, in all the assumed expanse of geologic time, is less than one chance out of a billion trillion. What possible conclusion, therefore, can we derive from such considerations as this except that evolution by mutation and natural selection is mathematically and logically indefensible!

Which has been (easily) debunked before:

This is a rather simplistic attack on evolution by mutation and natural selection. The way that he is calculating the probability actually does not take into account natural selection. What he is calculating is the chance that 200 random beneficial mutations happen sequentially without any bad or neutral mutations happening between the beneficial mutations. Natural selection will select and preserve the beneficial mutations and select against and eliminate the bad mutations. Once a beneficial mutation has become fixed in the population, any offspring produced that changed this would be selected against. His example also has no basis in a known biological system. His model essentially has an organism that reproduces only one offspring and then dies. All biological systems known to me the organisms on average have more than one offspring, allowing multiple chances for unique mutations to happen for selection to act on. Even using his unrealistic model slightly modified to have more than one offspring with adding in Natural Selection and having it take 2000 generations for a single beneficial mutation to fix in the population it would take ((2000 generations) * (200 fixed mutations)) = 400,000 generations to get the 200 fixed mutations, which with a generation time of 0.5 seconds would take about 2.3 days. If we use his numbers of one mutation per generation with a 1/2 probability of being beneficial, which would equal to a fixation rate of beneficial mutations about once every two generations much more generous than my 2000 generations. We do the same calculation with these numbers we get (2 generations) * (200 fixed mutations) = 400 generations, which is about 3 minutes. tl;dr: The guy is an idiot.

For a more academic discussion of this argument see "There’s plenty of time for evolution". The gist of that paper is

The fact is that with the parallel model, i.e., taking account of natural selection, the number of rounds of mutations that are needed to change the complete genome to its desirable form are only about K log L, instead of the hugely exponential K^L which would result from the serial model.

where L is the length of the genomic “word,” and K is the number of possible “letters” that can occupy any position in the word

For a broader treatment of information theory issues in ID vs evolution see Elsberry and Shallit; an older version of that paper can be found freely on teh interwebz.


Not the answer you're looking for? Browse other questions tagged .